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PROBLEM TO BE SOLVED: To provide a method and a 
device for image processing which can easily obtain a 
more adequate image by correcting distortion, and a 
recording medium which actualizes the method. 
SOLUTION: This is an image processor 1 which corrects 
distortion of images obtained by photographing a subject 
so that the images overlap with one another and is 
equipped with a correspondence detection part 21 which 
detects the correspondence relation between overlap 
parts in the images obtained by the photography, a 
reference image setting part 20 which selects an object 
of distortion correction out of the images, and an image 
distortion correction part 22 which corrects the distortion 
of the image set by the reference image setting part 20 

according to the correspondence relation detected by the correspondence detection part 21. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1 ] The first step which specifies the correspondence relation of the duplication part in said two or 
more images which are the image-processing approaches which amend distortion of the image photoed 
from two or more directions as at least a part overlaps to a photographic subject, and were obtained by 
said photography, The second step which chooses the object which amends said distortion from said two 
or more images, The image-processing approach characterized by having the third step which amends 
said distortion of said image chosen at said second step according to said correspondence relation 
specified in said first step. 

[Claim 2] The image-processing approach according to claim 1 which chooses the object of said 
amendment automatically at said second step according to the size of the field which said photographic 
subject occupies into said image. 

[Claim 3] The image-processing approach according to claim 1 which chooses the object of said 
amendment automatically at said second step according to the sense of the straight-line-like pattern 
detected in said image. 

[Claim 4] The image-processing approach according to claim 1 which chooses the object of said 
amendment automatically at said second step according to said correspondence relation specified in said 
first step. 

[Claim 5] The image-processing approach according to claim 1 which chooses the object of said 
amendment automatically at said second step according to the sense of said photographic subject 
detected for said every photography. 

[Claim 6] It is the image-processing approach which amends distortion of the image photoed from two 
or more directions so that at least a part might overlap to a photographic subject. The first step which 
specifies the correspondence relation of the duplication part in said two or more images obtained by said 
photography, respectively, The second step as which said distortion chooses said fewest images from 
said two or more images, The image-processing approach characterized by having said image which 
amended said distortion of said two or more images, respectively, and was chosen in said second step, 
and the third step to compound according to said correspondence relation specified in said first step. 
[Claim 7] A correspondence detection means to detect the correspondence relation of the duplication 
part in said two or more images which are the image processing systems which amend distortion of the 
image photoed from two or more directions as at least a part overlaps to a photographic subject, and 
were obtained by said photography, A selection means by which said distortion chooses said fewest 
images from said two or more images, The image processing system characterized by having said image 
which amended said distortion of said two or more images, respectively, and was chosen in said 
selection means, and an image composition means to compound, according to said correspondence 
relation detected in said correspondence detection means. 

[Claim 8] It is the image processing system which amends distortion of the image photoed from two or 
more directions so that at least a part might overlap to a photographic subject. A selection means to 
choose beforehand said image made into the criteria at the time of amending said distortion among said 
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two or more images to be photoed from now on, A notice means to notify a user of said image photoed 
next turning into said image made into said criteria according to said selection made by said selection 
means, A correspondence detection means to detect the correspondence relation of the duplication part 
between said image made into said criteria acquired by taking a photograph, and said other images, The 
image processing system characterized by having an amendment means to amend said distortion of said 
image made into said criteria, according to said correspondence relation detected by said correspondence 
detection means. 

[Claim 9] A correspondence detection means to detect the correspondence relation of the duplication 
part in said two or more images which are the image processing systems which amend distortion of the 
image photoed from two or more directions as at least a part overlaps to a photographic subject, and 
were obtained by said photography, The image processing system characterized by having a selection 
means to choose the object which amends said distortion from said two or more images, and an 
amendment means to amend said distortion of said image chosen by said selection means according to 
said correspondence relation detected by said correspondence detection means. 

[Claim 1 0] Two or more optical means which are the image processing systems which amend distortion 
of the image photoed from two or more directions as at least a part overlaps to a photographic subject, 
and photo said photographic subject to coincidence, A selection means to choose said image made into 
the object of said amendment among said images photoed by said two or more optical means, A 
correspondence detection means to detect the correspondence relation of the duplication part between 
said image chosen by said selection means, and said other images, The image processing system 
characterized by having an amendment means to amend said distortion of said selected image, according 
to said correspondence relation detected by said correspondence detection means. 
[Claim 11] Said selection means is an image processing system according to claim 9 or 1 0 which 
chooses the object of said amendment automatically according to the size of the field which said 
photographic subject occupies into said image. 

[Claim 12] Said selection means is an image processing system according to claim 9 or 10 which 
chooses the object of said amendment automatically according to the sense of the straight-line-like 
pattern detected in said image. 

[Claim 13] Said selection means is an image processing system according to claim 9 which chooses the 
object of said amendment automatically according to said correspondence relation detected by said 
correspondence detection means. 

[Claim 14] Said selection means is an image processing system according to claim 9 or 10 which 
chooses the object of said amendment automatically according to the sense of said photographic subject 
detected for said every photography. 

[Claim 1 5] It is the record medium which recorded the program for a computer to amend distortion of 
the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject and in which computer reading is possible. Said program makes the correspondence relation of 
the duplication part in said two or more images obtained by said photography specify to said computer. 
The record medium which is characterized by making said distortion of said image which was made to 
choose the object which amends said distortion and was chosen according to said specified 
correspondence relation from said two or more images amend and in which computer reading is 
possible. 

[Claim 1 6] It is the record medium which recorded the program for a computer to amend distortion of 
the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject and in which computer reading is possible. Said program makes the correspondence relation of 
the duplication part in said two or more images obtained by said photography specify to said computer, 
respectively. The record medium which is characterized by making it compound with said image with 
which said distortion made said fewest images choose, made said distortion of said two or more images 
amend according to said correspondence relation as which it was specified, respectively, and was chosen 
from said two or more images and in which computer reading is possible. 
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* NOTICES * 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is drawing for explaining the trouble in a Prior art. 

[Drawing 2] It is drawing for explaining the image-processing approach and image processing system 
concerning the gestalt 1 of operation of this invention. 

[Drawing 3] It is drawing showing the configuration of the image processing system concerning the 
gestalt 1 of operation of this invention. 

[Drawing 4] It is the perspective view showing the image processing system shown in drawing 3 . 
[Drawing 5] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 1 of operation of this invention. 

[Drawing 6] It is drawing showing the example of a display at the time of the photography in the display 
shown in drawing 3 . 

[Drawing 7] It is drawing showing the layout of the criteria image setting section shown in drawing 3 . 
[Drawing 8] It is drawing showing other examples of the image processing system concerning the 
gestalt 1 of operation of this invention. 

[Drawing 9] It is drawing showing the configuration of the correspondence detecting element shown in 



[Drawing 10] It is drawing explaining the actuation of correlation operation part shown in drawing 9 . 
[Drawing 11] It is drawing showing the configuration of the image-distortion amendment section shown 
in drawing 3 . 

[Drawing 12] It is drawing 1 explaining the actuation of the image-distortion amendment section shown 
in drawing 3 . 

[Drawing 13] It is drawing explaining the optical system of the image pick-up section shown in drawing 
3 . 

[Drawing 14] It is drawing explaining the projective transformation by the three-dimension operation 
part shown in drawing 1 1 . 

[Drawing 15] It is drawing 2 explaining the actuation of the image-distortion amendment section shown 
in drawing 3 . 

[Drawing 16] It is drawing 1 explaining the actuation of the parameter calculation section shown in 
drawing 1 1 . 

[Drawing 17] It is drawing 2 explaining the actuation of the parameter calculation section shown in 
drawing 1 1 . 

[Drawing 1 8] It is drawing for explaining the image-processing approach and image processing system 
concerning the gestalt 2 of operation of this invention. 

[Drawing 19] It is drawing showing the configuration of the image processing system concerning the 
gestalt 2 of operation of this invention. 

[Drawing 20] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 2 of operation of this invention. 

[Drawing 2 1 ] It is drawing showing the configuration of the image composition section shown in 



http://www4.ipdl.ncipi.go.jp/cgi-bin/tran_web_cgi_ejje 7/1 1/2006 



JP,2001-2501 14,A [DESCRIPTION OF DRAWINGS] 



Page 2 of 4 



drawing 19 . 

[Drawing 22] It is drawing showing the configuration of the image processing system concerning the 
gestalt 3 of operation of this invention. 

[Drawing 23] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 3 of operation of this invention. 

[Drawing 24] It is drawing explaining a setup of the criteria image to the criteria image setting section 
shown in drawing 22 . 

[Drawing 25] It is drawing explaining the actuation of the notice section shown in drawing 22 . 
[Drawing 26] It is drawing showing the configuration of the image processing system concerning the 
gestalt 4 of operation of this invention. 

[Drawing 27] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 4 of operation of this invention. 

[Drawing 28] It is drawing explaining a setup of the criteria image to the criteria image setting section 
shown in drawing 26 . 

[Drawing 29] It is drawing showing the first example of a configuration in the image processing system 
concerning the gestalt 5 of operation of this invention. 

[Drawing 30] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 5 of operation of this invention. 

[Drawing 31] It is drawing explaining the gate angular dependence in resolution degradation. 
[Drawing 32] It is drawing showing the second example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. 

[Drawing 33] It is drawing explaining the Hough conversion used in the image-processing approach 
concerning the gestalt 5 of operation of this invention. 

[Drawing 34] It is drawing showing the third example of a configuration in the image processing system 
concerning the gestalt 5 of operation of this invention. 

[Drawing 35] It is drawing showing the fourth example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. 

[Drawing 36] It is drawing showing the configuration of the flat-surface measurement section shown in 
drawing 35 . 

[Drawing 37] It is drawing showing the configuration of the image processing system concerning the 
gestalt 6 of operation of this invention. 

[Drawing 38] It is the flow chart which shows actuation of the image processing system concerning the 
gestalt 6 of operation of this invention. 

[Drawing 39] It is drawing showing the configuration of the image processing system concerning the 
gestalt 7 of operation of this invention. 

[Drawing 40] It is drawing which was shown in drawing 39 and which changes and explains actuation 
of the section. 

[Drawing 41] It is drawing showing the image processing system concerning the gestalt of operation of 
this invention, and a record medium. 

[Drawing 42] It is drawing showing the computer concerning the gestalt of operation of this invention, 
and the record medium in which computer reading is possible. 
[Description of Notations] 

60 1,2, 6, 8, 40, 50-53, 70 Image processing system 

3, 4, 10, IM1-IM3, iml, im2, imj, imk Image 

5 Distortion Amendment Image 

7 Criteria Image 

9 Reference Image 

1141 Image pick-up section 

11 A, 11B Optical system 

12 Signal-Processing Section 

1 3 Memory Control Section 
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14 Main Control Section 

1 5 Frame Memory 

16 Interface (I/F) 

1 7 Display 

1 8 Enternal Memory Section 

19 Photography Mode Setting Section 

20 Criteria Image Setting Section 

21 Correspondence Detecting Element 

22 Image-Distortion Amendment Section 

23 Photographic Subject Field Decision Section 

24 Image Composition Section 

25 Photographic Subject Field Judging Section 

26 Notice Section 

27 Criteria Image Automatic Selection Section 

28 Straight-Line-like Pattern Detecting Element 

29 Flat-Surface Measurement Section 

30 Image Side 

31 Plane of Projection-ed 

32 Image Pick-up Side 

33 Device-Coordinate System 
35 Photographic Subject Image 
45 Change Section 

101 Electric Power Switch 

102 Shutter 

103 Finder 

104 Photography Mode Setting Key 

111 Lens 

112 Drawing 

113 Shutter 

114 Optoelectric Transducer 

115 Pretreatment Section 

201 Above Scrolling Key 

202 Down Scrolling Key 

203 Decision Key 

204 Cursor Key 

211 Focus Setting Section 

212 Correlation Operation Part 

213 Focus 

215,216 Correlation aperture 
217 Corresponding Points 

221 Three-Dimension Operation Part 

222 Parameter Calculation Section 

223 Coordinate Transformation Section 

224 Image Side 

231 Projective-Transformation Calculation Section 

232 Coordinate Transformation Section 

241 Finder 

242 Indicator 

271 Spot Light Source 
271a Light source 
271b Scan mirror 
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271c Mechanical component 

272 Photo Detector 

273 Three-Dimension Coordinate Calculation Section 

274 Flat-Surface Calculation Section 

301 Record Medium 

302 CD-ROM 

PL Photographic subject side 

PC Personal computer (personal computer) 

D1-D3, dl, d2, dj, dk Direction 

IMA, 1MB, IMC Synthetic image 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the record medium in which computer reading for 
realizing the image-processing approach, the image processing system, and this image-processing 
approach for not being based on a photography condition but obtaining a proper image in more detail 
about the image-processing approach, an image processing system, and a record medium is possible. 
[0002] 

[Description of the Prior Art] It will be necessary to also diversify the state of business with the rapid 
advance of a computer network, and to acquire important information quickly on all aspects of affairs. In 
connection with it, the demand to inputting goods required for business and document information 
simple and with high definition everywhere, making full use of the input device of a pocket mold is 
increasing. Especially, the application that it will utilize as electronic intelligence effective in work or 
amusement also came to be seen by performing processing and processing to the photoed image with 
rapid spread and its raise in resolving of a digital still camera. 

[0003] Although the distortion of an image (this is also called "gate distortion".) arises as the first typical 
application when photoing photographic subject sides, such as A4 space and a large-sized poster, and an 
image pick-up side and a photographic subject side take a photograph in the condition which is not 
parallel, there is a technique of raising the readability of the document image information acquired by 
amending this. Here, in the image which wants to amend "gate distortion", it is required that the range of 
the request of a photographic subject should be copied. That is, it is necessary to photo at least one in the 
image of two or more sheets which the user photoed so that the range of the request of a photographic 
subject may be included. Therefore, the interface (I/F) which can choose the object which amends gate 
distortion among the photoed images of two or more sheets, and an interface to which suitable cautions 
are urged to a user at the time of photography are desired. 

[0004] Moreover, there are some which create the synthetic image of one sheet by carrying out division 
photography of the pattern drawn on the space information and the panels of a large area, such as a 
newspaper, or a wall by the portable picture input device as the second application, and sticking the 
obtained image of two or more sheets. That is, although the resolution of a digital camera is improving 
in recent years with the increment in the number of pixels of the image sensor represented by CCD 
(Charge Coupled Device), photoing the photographic subject which has a pattern fine as mentioned 
above, i.e., the photographic subject containing a high frequency component, and electronizing runs 
short of resolution still more. Therefore, by sticking an image, a highly minute image is created in false 
and the approach of compensating the lack of resolution of a digital camera is made. 
[0005] When a photographic subject can regard it as a plane, a geometric correction type like affine 
transformation or projective transformation is used, it is the technique which sticks the image which 
carried out division photography of some photographic subjects, and such an application changes the 
photographic subject image of each division image into how the photographic subject in the image used 
as criteria appears, and sticks. In addition, the outline of such a technique is indicated by reference 
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"computer vision-technical criticism and future view -" (new technical communique SHONZU besides 
Takashi Matsuyama). 

[0006] However, in such an application, when it instigates in a photographic subject image in the image 
used as criteria and distortion has arisen, it instigates also in the stuck synthetic image and there is a 
problem that distortion will be included. That is, the magnitude of the gate distortion in the synthetic 
image generated changes by on the basis of which image lamination is performed among two or more 
images which took a photograph in division and were obtained. It explains referring to drawing 1 about 
this problem. 

[0007] If these three images are stuck on the basis of the image IM 1 photoed and obtained from left 
slant to the photographic subject side PL when a certain photographic subject side PL is photoed from 
three ways D1-D3 and images IM1-IM3 are obtained, respectively, as shown in drawing 1 (a) for 
example, the synthetic image IMA as shown in drawing 1 (b) will be obtained. Moreover, if it sticks 
similarly on the basis of the photoed image IM 2 from a transverse plane mostly, the synthetic image 
1MB as shown in drawing 1 (c) will be obtained. Here, as shown in drawing 1 (b) and drawing 1 (c), 
both the above-mentioned composition images IMA and 1MB differ in the magnitude of gate distortion 
greatly. 

[0008] Therefore, since a photograph is wanted to carry out the right pair of at least one in the photoed 
image of two or more sheets to the photographic subject side PL mostly, and to be taken, in case it is 
lamination among the photoed images of two or more sheets, when photoing the interface which 
chooses the image made into criteria, and the image used as the criteria of the lamination, an interface 
which demands cautions from a user is desired. 
[0009] 

[Problem(s) to be Solved by the Invention] This invention is made in view of an above-mentioned point, 
and aims at offering the record medium in which computer reading for realizing the image-processing 
approach, the image processing system, and this image-processing approach for amending distortion and 
obtaining a more proper image easily is possible. 
[0010] 

[Means for Solving the Problem] The above-mentioned purpose is the image-processing approach which 
amends distortion of the image photoed from two or more directions so that at least a part might overlap 
to a photographic subject. The first step which specifies the correspondence relation of the duplication 
part in two or more images obtained by photography, The second step which chooses the object which 
amends distortion from two or more images, It is attained by offering the image-processing approach 
characterized by having the third step which amends distortion of the image chosen at the second step 
according to the correspondence relation specified in the first step. Since the object which amends 
distortion is chosen according to such a means, in order to obtain a proper image, the optimal image can 
be made applicable to amendment. 

[001 1] You may make it choose the object of amendment automatically at the second step here 
according to the size of the field which a photographic subject occupies into an image. According to 
such a means, the paintings-and-calligraphic-works amount of information needed can amend distortion 
of the most abundant images automatically. 

[0012] Moreover, you may make it choose the object of amendment automatically at the second step 
according to the sense of the straight-line-like pattern detected in an image. According to such a means, 
the image photoed from the location which carried out the right pair to the photographic subject mostly 
can be automatically made applicable to amendment. 

[0013] Moreover, at the second step, if the object of amendment is automatically chosen according to 
the correspondence relation specified in the first step, high amendment of precision can be performed 
certainly. 

[0014] Moreover, at the second step, if the object of amendment is automatically chosen according to 
the sense of the photographic subject detected for every photography, the image photoed from the 
location which carried out the right pair mostly to the photographic subject can be automatically made 
applicable to amendment. 
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[0015] Moreover, the purpose of this invention is the image-processing approach which amends 
distortion of the image photoed from two or more directions so that at least a part might overlap to a 
photographic subject. The first step which specifies the correspondence relation of the duplication part 
in two or more images obtained by photography, respectively, The second step as which distortion 
chooses fewest images from two or more images, According to the correspondence relation specified in 
the first step, distortion of two or more images is amended, respectively, and it is attained by offering the 
image-processing approach characterized by having the image chosen in the second step, and the third 
step to compound. According to such a means, since a synthetic image is generated on the basis of an 
image with least distortion, a more proper synthetic image can be obtained. 

[0016] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A correspondence detection means to detect the correspondence relation of the duplication part 
in two or more images obtained by photography, It responds to the correspondence relation by which 
distortion was detected in a selection means to choose fewest images, and the correspondence detection 
means, out of two or more images. Distortion of two or more images is amended, respectively, and it is 
attained by offering the image processing system characterized by having the image chosen in the 
selection means, and an image composition means to compound. 

[0017] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A selection means to choose beforehand the image made into the criteria at the time of 
amending distortion among two or more images to be photoed from now on, A notice means to notify a 
user of the image photoed next turning into an image made into criteria according to the selection made 
by the selection means, A correspondence detection means to detect the correspondence relation of the 
duplication part between the image made into the criteria acquired by taking a photograph, and other 
images, It is attained by offering the image processing system characterized by having an amendment 
means to amend the distortion of an image made into criteria, according to the correspondence relation 
detected by the correspondence detection means. According to such a means, in the photography of an 
image made into the criteria at the time of amending distortion, attention can be called to a user. 
[0018] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A correspondence detection means to detect the correspondence relation of the duplication part 
in two or more images obtained by photography, It is attained by offering the image processing system 
characterized by having a selection means to choose the object which amends distortion from two or 
more images, and an amendment means to amend distortion of the image chosen by the selection means 
according to the correspondence relation detected by the correspondence detection means. 
[0019] Moreover, two or more optical means which the purpose of this invention is an image processing 
system which amends distortion of the image photoed from two or more directions as at least a part 
overlaps to a photographic subject, and photo a photographic subject to coincidence, A selection means 
to choose the image made into the object of amendment among the images photoed by two or more 
optical means, A correspondence detection means to detect the correspondence relation of the 
duplication part between the image chosen by the selection means, and other images, It is attained by 
offering the image processing system characterized by having an amendment means to amend distortion 
of the selected image, according to the correspondence relation detected by the correspondence detection 
means. Since two or more photographic subject images once photoed by photography from two or more 
directions can be obtained according to such a means, the count of photography needed in order to 
amend distortion can be reduced. 

[0020] Here, a selection means shall choose the object of amendment automatically according to the 
sense of the straight- line-like pattern which shall choose the object of amendment automatically 
according to the size of the field which a photographic subject occupies into an image, or is detected in 
an image. Moreover, a selection means is good also as what chooses the object of amendment 
automatically according to the sense of the photographic subject which should choose the object of 
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amendment automatically according to the correspondence relation detected by the correspondence 
detection means, or was detected for every photography. 

[0021] Moreover, the purpose of this invention is a record medium which recorded the program for a 
computer to amend distortion of the image photoed from two or more directions so that at least a part 
might overlap to a photographic subject and in which computer reading is possible. This program makes 
the correspondence relation of the duplication part in two or more images obtained by photography 
specify to a computer. It is attained by offering the record medium which is characterized by making 
distortion of the image which was made to choose the object which amends distortion and was chosen 
according to the specified correspondence relation from two or more images amend and in which 
computer reading is possible. According to such a means, in order to obtain a proper image, the optimal 
image can be easily made applicable to amendment. 

[0022] Moreover, the purpose of this invention is a record medium which recorded the program for a 
computer to amend distortion of the image photoed from two or more directions so that at least a part 
might overlap to a photographic subject and in which computer reading is possible. This program makes 
the correspondence relation of the duplication part in two or more images obtained by photography 
specify to a computer, respectively. It is attained by offering the record medium which is characterized 
by making it compound with the image which was made to amend distortion of two or more images, 
respectively, and was chosen according to the correspondence relation as which distortion made fewest 
images choose as and it was specified from two or more images and in which computer reading is 
possible. According to such a means, a more proper synthetic image, can be obtained easily. 
[0023] 

[Embodiment of the Invention] The gestalt of operation of this invention is explained in detail with 
reference to a drawing below. In addition, a same-among drawing sign shows the same or a considerable 
part. 

[Gestalt 1 of operation] drawing 2 is drawing for explaining the image-processing approach and image 
processing system concerning the gestalt 1 of operation of this invention. As the gestalt of this operation 
is shown to drawing 2 by here, the images 3 and 4 of two sheets are photoed so that a part of 
photographic subject side [ at least ] PL may overlap with an image processing system 1, and it is 
explained by the example of amending the gate distortion by either 3, for example, an image, among the 
images 3 and 4 of these two sheets, and finally obtaining the distortion amendment image 5. 
[0024] Drawing 3 is drawing showing the configuration of the image processing system 1 concerning 
the gestalt 1 of operation of this invention. As shown in drawing 3 , an image processing system 1 is 
equipped with the image pick-up section 1 1 , the signal-processing section 1 2, the memory control 
section 13, the main control section 14, a frame memory 15, an interface 16, a display 17, the enternal 
memory section 18, the photography mode setting section 19, the criteria image setting section 20, the 
correspondence detecting element 21, and the image-distortion amendment section 22. And the image 
pick-up section 1 1 contains a lens 111, diaphragm 1 12, a shutter 1 13, an optoelectric transducer 114, 
and the pretreatment section 115. 

[0025] Here, the signal-processing section 12 is connected to the pretreatment section 115, the memory 
control section 13, the main control section 14, and an interface 16. Moreover, the memory control 
section 13 is further connected to a frame memory 15 and the criteria image setting section 20. The main 
control section 14 is further connected to the memory control section 13, the photography mode setting 
section 19, and the criteria image setting section 20. 

[0026] Moreover, a frame memory 15 is connected to the memory control section 13, the 
correspondence detecting element 21, and the image-distortion amendment section 22. Moreover, an 
interface 16 is further connected to a display 17 and the enternal memory section 18. And the criteria 
image setting section 20 is further connected to the image-distortion amendment section 22. Moreover, 
the correspondence detecting element 2 1 is further connected to the image-distortion amendment section 



[0027] On the other hand, in the image pick-up section 1 1, it extracts as a lens 111, and 1 12, a shutter 
113, and an optoelectric transducer 114 are arranged on an optical axis at this order, and an optoelectric 



22. 
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transducer 1 14 is connected to the pretreatment section 115. 

[0028] In the above, photography mode is changed by the photography mode setting section 1 9, and the 
image which instigates in the criteria image setting section 20, and amends distortion is set up. 
Moreover, the correspondence detecting element 2 1 extracts the focus and corresponding points between 
both images in two images with which at least the part overlapped mutually. And the image-distortion 
amendment section 22 amends the gate distortion in the photoed image according to the signal supplied 
from the correspondence detecting element 2 1 . In addition, a setup of the criteria image in the above, 
actuation of the correspondence detecting element 2 1 , and amendment of gate distortion are explained in 
detail later. 

[0029] Moreover, CCD is used for the optoelectric transducer 1 14 of the image pick-up section 11. 
Moreover, the pretreatment section 1 1 5 is equipped with the analog signal processing section and the 
analog-digital converter (A/D converter) which consist of pre amplifier, an automatic gain control circuit 
(Auto Gain Control- AGC), etc., and after pretreatment of magnification, a clamp, etc. is performed to 
the analog video signal outputted from the optoelectric transducer 114, the above-mentioned analog 
video signal is changed into a digital video signal. 

[0030] Moreover, the signal-processing section 12 is constituted by the digital signal processor (DSP 
processor) etc., and performs various image processings, such as color separation, white balance 
adjustment, and gamma amendment, to the digital video signal acquired in the image pick-up section 1 1 . 
Moreover, the memory control section 1 3 stores in a frame memory 1 5 the picture signal processed by 
doing in this way, or reads the picture signal conversely stored in the frame memory 15. Moreover, the 
main control section 14 is constituted by the microcomputer etc. Moreover, a frame memory 15 stores 
the image of at least two sheets, and, generally semiconductor memory, such as VRAM, SRAM, and 
DRAM, is used. 

[0031] Here, the picture signal read from the frame memory 15 is saved in the entemal memory section 
18 through an interface 16, after signal processing, such as picture compression, is performed in the 
signal-processing section 12. Reading and this entemal memory section 18 write various signals, such as 
a picture signal supplied through an interface 16, and is constituted by IC memory card, the magneto- 
optic disk, etc. As the entemal memory section 18, if a modem card and an ISDN card are used, a 
picture signal can also be transmitted to the record medium of a direct remote place via a network here. 
[0032] Moreover, a picture signal is transmitted to the signal-processing section 12 through an interface 
1 6, and read-out of the picture signal conversely recorded on the entemal memory section 1 8 is 
performed by giving image expanding in the signal-processing section 12. On the other hand, the display 
of the picture signal read from the entemal memory section 18 and a frame memory 15 is performed by 
transmitting to a display 17 through an interface 16, after performing signal processing, such as digital 
to analog (D/A conversion) and magnification, to a picture signal in the signal-processing section 12. A 
display 1 7 consists of liquid crystal displays which displayed the image according to the picture signal 
supplied through the interface 16, for example, were installed in the case of an image processing system 
1 here. 

[0033] Drawing 4 is the perspective view showing the image processing system shown in drawing 3 . As 
shown in drawing 4 , the image processing system 1 concerning the gestalt of this operation contains an 
electric power switch 101, a shutter 102, a finder 103, the photography mode setting key 104 for setting 
photography mode as the photography mode setting section 19, the above scrolling key 201 for scrolling 
upward the image projected on the display 17, the down scrolling key 202 for scrolling downward the 
image projected on the display 17, and the decision key 203. 

[0034] It explains referring to the flow chart shown in drawing 4 and drawing 5 in actuation of the 
image processing system which has the above configurations below. First, an electric power switch 101 
is changed, an image processing system 1 is started, and photography mode is chosen. Here, the above- 
mentioned photography mode consists of the normal mode which takes the usual snapshot, and gate 
amendment mode which generates the image which amended the gate distortion of the photoed image. 
And selection in this photography mode is made when a user operates the photography mode setting key 
104. In addition, as the photography mode setting section 19, the photography mode setting key 104 is 
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formed in the body of an image processing system 1 . However, the photography mode setting section 1 9 
may consist of hardware or software etc. which are formed separately from a body. 
[0035] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 5 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0036] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And a user photos at least two photographic subject sides PL by the image pick-up section 11, 
and a photographic subject image is incorporated by the image processing system 1. In addition, each 
image needs to be photoed so that some photographic subject images may overlap mutually at this time. 
[0037] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. In addition, the above-mentioned 
directions are made by pressing the photography mode setting key 1 04 once again, and changing to the 
normal mode, and also they may form the switch for directing photography termination separately. 
[0038] Moreover, as shown during photography at drawing 6 , the screen overlay of the present 
photography number of sheets, such as "the 1st sheet", and the photography termination approaches, 
such as "ending, if a photography mode setting key is pressed", may be carried out to a display 17. 
[0039] And although the actuation which instigates below to step S4 and amends distortion is started 
after photography is completed as mentioned above, it chooses the gate distortion of which image is first 
amended in step S4, and this image is set up. In addition, the image chosen at this time is also called a 
"criteria image" to below. And a setup of this criteria image is performed in the criteria image setting 
section 20. The configuration and actuation of the criteria image setting section 20 are explained in 
detail below. 

[0040] Drawing 7 is drawing showing the layout of the criteria image setting section 20 shown in 
drawing 3 . As shown in drawing 7 , the criteria image setting section 20 contains the above scrolling 
key 201 , and the down scrolling key 202 and the decision key 203. And as shown in drawing 7 , when 
directions of photography termination are made by the user in step S3, in a display 17, an overlay 
indication of the alphabetic character "criteria image setup" is given, and it points so that a criteria image 
may be chosen as a user. 

[004 1 ] Then, the above scrolling key 20 1 and the down scrolling key 202 are operated by the user, and it 
is displayed, the image photoed in the above-mentioned step S2 changing one by one. In addition, the 
image photoed after [ of the image by which it is indicated by current ] one is displayed by pressing the 
above scrolling key 201 by displaying the image photoed before [ of the image by which it is indicated 
by current ] one, and pressing the down scrolling key 202. And if the decision key 203 is pressed in the 
condition that the image chosen as a criteria image is displayed on the display 1 7, the image currently 
displayed at the time will be determined as a criteria image. 

[0042] And the image with the smallest possible angle that crossed to the field where the photographic 
subject side PL is the largest, and has been reflected to it in two or more photoed images as the above- 
mentioned criteria image, and is made to the photographic subject side of the image pick-up side at the 
time of photography is chosen by the user. 

[0043] Here, choosing automatically the image optimal as the object which calculates a photographic 
subject field and the above-mentioned tilt angle inside equipment, instigates according to the result of 
this count, and amends distortion, i.e., a criteria image, is also considered. Drawing 8 is drawing 
showing the configuration of the image processing system 2 which realizes such actuation. As shown in 
drawing 8 , an image processing system 2 is equipped with the photographic subject field decision 
section 23 instead of the criteria image setting section 20 contained in the image processing system 1 
shown in drawing 3 . 

[0044] And as the above-mentioned photographic subject field decision section 23 performs processing 
which detects the field which a photographic subject occupies in the photoed image, for example, it is 
indicated by reference "processing of an image and recognition" (Takeshi Akoin and Tomoharu Nagao 
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collaboration, Shokodo) (a) How to cluster on an image like a field grown method or a field split plot 
experiment, (b) The field division approaches using the edge in images, such as the approach of 
clustering on feature spaces, such as field division by the histogram, and (c) profile line tracking, such as 
an approach and the (d) texture analysis, are applied. However, when a photographic subject side is a 
rectangle, the field of a photographic subject is determined as a meaning by carrying out the external 
input of the top-most- vertices coordinate of the four corners of the photographic subject on an image. 
[0045] Thus, the data in which the field of the photographic subject acquired in the photographic subject 
field decision section 23 is shown are supplied to the image-distortion amendment section 22, and an 
image with this largest field is chosen in the image-distortion amendment section 22. Furthermore, in the 
image-distortion amendment section 22, since the include angle made to the photographic subject side 
PL of an image pick-up side is calculated, when there are two or more images with the above-mentioned 
largest field (i.e., when there are two or more images with which for example, the whole photographic 
subject side is reflected), finally an image with the above-mentioned smallest include angle is chosen as 
a criteria image, so that it may explain in full detail below. 

[0046] In addition, the above functions which choose a criteria image automatically are applicable 
similarly in the gestalt of which [ of the following ] operation. 

[0047] Next, it progresses to step S5, the focus is detected in the criteria image determined as mentioned 
above, and the corresponding points which show the same part as the above-mentioned focus in the 
image (it is also called a "reference image" to below.) which overlaps in these some criteria images [ at 
least ] are detected. Detection of such the focus and corresponding points is performed by the 
correspondence detecting element 2 1 shown in drawing 3 . Then, the configuration and actuation of this 
correspondence detecting element 21 are explained in detail below. 

[0048] The correspondence detecting element 21 detects the photoed same part in the image with the 
field which overlapped mutually as mentioned above of two sheets. And the approach which used the 
correlation operation here is explained. 

[0049] Drawing 9 is drawing showing the configuration of the correspondence detecting element 2 1 
shown in drawing 3 . As shown in drawing 9 , the correspondence detecting element 2 1 is equipped with 
the focus setting section 211 connected to the frame memory 15, and the correlation operation part 212 
connected to the focus setting section 211 and a frame memory 15. In addition, the criteria image and 
the reference image are stored in the frame memory 1 5 shown in drawing 3 . 

[0050] After the focus setting section 211 determines the location of the focus in a criteria image, it 
extracts the shade pattern of x (2N+1) (2P+1) individual centering on the focus, and creates the data of 
the field called a correlation aperture here. In addition, the location of the above-mentioned focus is 
determined by extracting the part where the concentration pattern of an image is characteristic like an 
angle (comer). 

[0051] Moreover, the correlation operation part 212 detects the part which is mostly in agreement with 
the shade pattern of the correlation aperture created based on the criteria image by performing a 
correlation operation in a reference image, and determines this as corresponding points. An example 
which detects corresponding points by block matching by the correlation operation here is explained 
referring to drawing 10 . 

[0052] As shown in drawing 10 , in block matching of the correlation aperture 215,216 which consists 
of a shade pattern of x (2N+1) (2P+1) individual, the cross-correlation value Si of the i-th focus 213 
which has a coordinate in the criteria image 7 (xiO, yiO), and the corresponding points 217 which have a 
coordinate in the reference image 9 (xiO+dxi, yiO+dyi) is calculated by the degree type. 
[0053] 
[Equation 1] 
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In addition, in the above-mentioned formula (1), Is (x y) shows concentration [ in / for the concentration 
in the coordinate point (x y) of the criteria image 7 / in an example and Ir (x y) / the coordinate point (x 
y) of the reference image 9 ]. Moreover, Is (x y) shows the average concentration in the pattern of x 
(2N+1) (2P+1) individual centering on the coordinate point in the /correlation aperture 215 in the criteria 
image 7 (x y). Ir (x y) shows the average concentration in the pattern of x (2N+1) (2P+1) individual 
centering on the coordinate point in the /correlation aperture 216 in the reference image 9 (x y). 
Moreover, K shows a constant. 

[0054] And the corresponding points 217 in the reference image 9 are called for by searching for the 
point which is beyond the threshold as which the maximum of the cross-correlation value Si was 
beforehand determined by the above-mentioned formula (1) to each focus 213. In addition, if the 
maximum of the cross-correlation value Si becomes below a threshold, corresponding points shall not 
exist. 

[0055] Thus, after detection of the focus and corresponding points is completed, while calculating the 
parameter which amends the gate distortion of the criteria image 7 in step S6 shown in drawing 5 , the 
image which amended the gate distortion of an image based on this parameter in step S7 is created, and 
actuation is ended. In addition, the image which amended a "distortion amendment parameter" and gate 
distortion for the above-mentioned parameter below, respectively is also called "distortion amendment 
image." 

[0056] And count of the above-mentioned distortion amendment parameter and generation of a 
distortion amendment image are performed by the image-distortion amendment section 22. The 
configuration and actuation of this image-distortion amendment section 22 are explained in detail below. 

[0057] The image-distortion amendment section 22 amends gate distortion by changing into the image 
which photoed the photographic subject side from the transverse plane using the relation of the focus 
and corresponding points which the correspondence detecting element 2 1 detected. And the 
configuration of this image-distortion amendment section 22 is shown in drawing 1 1 . As shown in 
drawing 1 1 , the image-distortion amendment section 22 contains the three-dimension operation part 
221, the parameter calculation section 222, and the coordinate transformation section 223. Here, the 
three-dimension operation part 221 is connected to the correspondence detecting element 21 and the 
criteria image setting section 20, and the parameter calculation section 222 is connected to the three- 
dimension operation part 221. Moreover, the coordinate transformation section 223 is connected to the 
parameter calculation section 222, a frame memory 15, and the criteria image setting section 20. 
[0058] Actuation of the image-distortion amendment section 22 is explained below. In addition, as 
shown below at drawing 12 , while the criteria image 7 and the reference image 9 are photoed to the 
photographic subject side PL As the optical system of the image pick-up section 1 1 is shown in drawin g 
13 , it is related with a x axis. Facing the right of the image side 224 Forward, The sense which goes 
facing down of the image side 224 to the image side 224 about the z-axis of forward and the direction of 
an optical axis about the y-axis from the zero O which is the optical axis of the image pick-up section 1 1 
Forward, The focal distance of this optical system explains as an example the case where it considers as 
the central projection model (perspectiveprojection model) set to f. 

[0059] The three-dimension operation part 221 shown in drawing 1 1 computes the following three- 
dimension parameters with the relation between the above-mentioned focus 213 and corresponding 
points 217. That is, normal vector n which shows the sense of the translational-motion vector t which 
shows change of the location of the image pick-up section 1 1 at the time of the reference image 
photography based on the time of the rotation matrix R which shows change of the sense of the image 
pick-up section 1 1 at the time of the reference image photography based on the time of the criteria image 
photography shown in drawing 12 , and criteria image photography, and the photographic subject side 
PL is computed. And the method of asking for these three three-dimension parameters {R, t, n} is 
mainly made into two of the followings. 

[0060] that is, as a primary method, by the group of the eight or more focus and corresponding points, 
after calculating the location of the camera at the time of each image photography, a posture, and the 
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three-dimension coordinate of each corresponding points, it assumes that a photographic subject is a flat 
surface, and there is a method of applying this profit **** three-dimension coordinate to one flat 
surface. 

[0061] Moreover, as the second approach, from the group of the four or more focus and corresponding 
points, a projective-transformation matrix (homography matrix) is calculated and there is the approach 
of computing the location of the camera at the time of each image photography and the sense of a 
posture and a photographic subject side according to the acquired projective-transformation matrix. 
[0062] Here, although the above-mentioned primary method is a general-purpose movement 
stereoscopic vision technique and a meaning is asked for the above-mentioned parameter {R, t, n} by 
the linearity operation, the detail is recorded on the common reference (for example, "three-dimension 
vision" **** and crossing Saburo collaboration, KYORITSU SHUPPAN) about three-dimension 
measurement or a computer vision. On the other hand, the second approach of the above computes the 
sense of a camera, and the sense of a photographic subject side, after asking for the coordinate 
transformation equation (projective-transformation matrix) materialized under the constraint that a 
photographic subject is a flat surface. And although the three-dimension operation part 221 can, needless 
to say, take any [ the above-mentioned first and / second ] approach, it explains the actuation based on 
the second approach here. 

[0063] The calculation procedure of a projective-transformation matrix is explained in detail first. Here, 
it points out obtaining the image 10 changed into the image obtained when the photographic subject 
image reflected to the criteria image 7 is photoed from the same direction as the reference image 9, as 
indicated in drawing 14 as the projective transformation from a criteria image to a reference image. And 
when this projective transformation is expressed with a formula and the point (xs, ys) in a criteria image 
and the point (xr, yr) in a reference image have a correspondence relation, it becomes like a degree type. 
[0064] 
[Equation 2] 



And they are eight unknowns bl-b8 in the above-mentioned formula (2) [0065] 
[Equation 3] 



1 



C3) 



It collects as a matrix B and this is called a projective-transformation matrix. In order to search for this 
projective-transformation matrix B, 4 or more sets of groups of the focus whose coordinates in a criteria 
image are (xsi, ysi), and the corresponding points (i= 1 , — , N;N>=4) whose coordinates in a reference 
image are (xri, yri) are used. Here, although what is necessary is to substitute a coordinate (xsi, ysi) and 
a coordinate (xri, yri) to the above-mentioned formula (2), and just to calculate the solution of bl-b8, 
since a formula (2) is not materialized according to errors, such as a noise superimposed on an image in 
fact, it will solve using the least square operation shown below. 
[0066] 
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And the above-mentioned formula (4) deforms as follows. 
[0067] 
[Equation 5] 



E £(^, + ^ + by ~ {b,X M + ^ -f- \)x rt }* 

imi 

+[b A x ti + btf,, +b 6 - fax,, + bo?* + l)y H ) 2 3 -> min. ' 



(5) 



Use of the constraint that the value of the derivative obtained by carrying out strange differential of the 
left part of the above-mentioned equation (5) by bl-b8, respectively is set to 0 calculates bl-b8 by 
solving a simultaneous equation. That is, the projective-transformation matrix B can be searched for by 
the easy linearity operation using the group matched the account of a top. 

[0068] Then, the procedure of asking for a three-dimension parameter {R, t, n} from the projective- 
transformation matrix B is explained. It is normal vector n of a photographic subject side [0069] 
[Equation 6] 



b 
c 
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(6) 



It is the equation of the photographic subject [ carry out and ] side on the basis of the time of criteria 
image photography [0070] 
[Equation 7] 

(/*,r)+d = 0 (7) 

Here, |d| shows the distance from the zero to the photographic subject side PL, and is r=[x y z] T. It sets. 
Moreover, a formula (2) is rewritten like a degree type using a focal distance f. 
[0071] 
[Equation 8] 

= . H^x^H^y^HyJ 
y ' J H»x s +Hvy, + H»f 



Furthermore, a formula (8) is [0072]. 
[Equation 9] 
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It deforms. However, [0073] 
[Equation 10] 
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It comes out. At this time, the matrix H of a formula (9) and the relation with a parameter {R, t, n, d} 

become like a degree type. 

[0074] 

[Equation 11] 

/ ,i ( u > 

H T = s'R T \dI + ln T ) 

However, s' is a constant and each element of Matrix H has one times the degree of freedom of a scale. 
Moreover, a degree type can perform conversion in procession H of a formula (9) from the projective- 
transformation matrix B of a formula (3). 
[0075] 



T 0 


0" 




"1 


0 


0 " 












0 I 


0 


B 


0 


I 


0 








V/ 


(12) 


0 0 


/. 




0 


0 


*//. 




A 


A 


1 





The procedure of asking below for a parameter {R, t, n, d} stranger than Matrix H is shown. However, 
since it is unfixed, Variable d and the scale of the magnitude of the translational-motion vector t 
showing distance with a photographic subject side are [0076]. 
[Equation 13] 

(ltll=l (13) 

It assumes. Here, although the detail is describing in reference "mathematical principle of image 
comprehension-three-dimension recognition -" (Ken-ichi Kaneya work, Morikita Shuppan), the 
computation which derives the solution {R, t, n, d} to is as follows when the result is summarized, 
(i) It is made to be set to det[H] =1, applying the suitable constant for each element of the matrix H of a 
formula (9). (ii) Characteristic value of a symmetric matrix HHT is set to sigma!2, sigma22, and 
sigma32, the corresponding characteristic vectors ul, u2, and u3 are mutually intersected 
perpendicularly, and it takes to the unit vector which makes a right-hand system in this order. However, 
sigmal>=sigma2>=sigma3> It is referred to as 0. (iii) A movement parameter will be [0077] if it 
becomes sigmal=sigma2=sigma3. 
[Equation 14] 

t = 0, R = H (14) 

It comes out, and it is and the parameter {n, d} of a photographic subject side is unfixed. Otherwise, 2 
sets of solutions are acquired as follows, (iv) The parameter {n, d} of a photographic subject side 
becomes settled as follows. 
[0078] 

[Equation 15] 



n = 
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(15) 



[0079] 

[Equation 16] 
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(16) 



However, it is epsilon= **1, and epsilon is chosen so that it may be set to c> 0. Moreover, (v) unit 

translational-motion vector t becomes settled as follows. 

[0080] 
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[Equation 17] 
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(17) 



Moreover, the rotation matrix R becomes settled as follows. 
[0081] 

[Equation 18] 



{n 9 t)+d J 



(18) 



Therefore, although two kinds of solutions {R, t, n, d} to are acquired from Matrix B, when the most, a 
true solution can be distinguished from the drawn value. In addition, although the focal distance f of the 
image pick-up section 1 1 is used by intermediate count, the value of a focal distance f can be easily 
obtained by the approach of memorizing the optical- system parameter of the image pick-up section 1 1 to 
an internal memory (not shown). Moreover, if the focal distance of both images is known even when the 
focal distances of a criteria image and a reference image differ [ the focal distance of the optical system 
of the image pick-up section 11] with adjustable, the above-mentioned three-dimension parameter 
calculation procedure can be applied as it is, and a focal distance can be detected by the approach of 
installing an encoder in optical system. 

[0082] Next, the parameter calculation section 222 shown in drawing 1 1 calculates the parameter which 
amends gate distortion based on relation with the sense of the photographic subject side computed by 
the image pick-up section 1 1 and the three-dimension operation part 221 when photoing a photographic 
subject side. As the gestalt of this operation is shown to drawing 15 by here, the gate distortion of an 
image is amended by performing projective transformation of projecting the image side 30 containing 
the photographic subject image which has gate distortion, by using as the plane of projection 31-ed a flat 
surface parallel to the photographic subject side which the three-dimension operation part 221 
computed. The point PI on the image side 30 is projected on the point P2 on the plane of projection 31- 
ed here. The count approach of the parameter which amends the above-mentioned gate distortion below 
is explained. 

[0083] First, as shown in drawing 16 , it asks for rotation matrix R f which shows the coordinate 
transformation which makes the z-axis in the device-coordinate system 33 in agreement with the unit 
normal vector of the plane of projection 31-ed. In this case, the following relational expression is 
materialized. 
[0084] 





0" 
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R' 


0 




b 




1 




c 

-* — 



(19) 



And although much rotation matrix R' which fills the above-mentioned formula (9) exists, rotation 

matrix R 1 is defined like a degree type here. 

[0085] 

[Equation 201 



(20) 
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However, R f x and R f y are shown as follows here, respectively. 
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[0086] 

[Equation 21] 
1 0 



0 cosa 
0 sina 
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-sina 
cosa 



K = 



cos/3 
0 



0 


sin/3 
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0 


0 


cos/? 



(21) 



As shown in drawing 17 , this rotates the device-coordinate system (xyz system of coordinates) 33 in 
following sequence, and is equivalent to changing into x'y'z 1 system of coordinates, (i) beta rotation of 
the device-coordinate system 33 is done only around the y-axis. And let the system of coordinates 
obtained by this rotation be lylzxl system of coordinates, (ii) Only alpha rotates a device-coordinate 
system around x 1 shaft. 

[0087] Here, if a formula (19) and a formula (20) are used, an angle of rotation will be drawn like a 
degree type. 
[0088] 
[Equat 
a 



quation 221 
= sin- 1 (- b) 



(22) 



(23) 



And matrix R' can be set to a meaning by substituting for a formula (20) and a formula (21) the angle of 
rotation searched for in this way. 

[0089] Next, coordinate transformation of the coordinate on the image side 30 is carried out on the plane 
of projection 31-ed. That is, let the point P2 which intersects the plane of projection 31 -ed when the 
three-dimension vector p corresponding to the point PI of the image side 30 is extended be a coordinate 
after coordinate transformation in drawing 15 . And the three-dimension vector p corresponding to the 
point PI on the basis of the device-coordinate system 33 is shown by the degree type. 
[0090] 

[Equation 23] 

k 



L/J 



(*>0) 



(24) 



Here, since k is a scaling factor showing the distance from the optical axis o of the image pick-up 
section 1 1 to plane of projection-ed, k expresses the magnitude of the distortion amendment image 
created, moreover — a three dimension — a vector — p — criteria — an image — photography — the time — 
a device coordinate — a system — 33 — criteria — carrying out — a point — P — one — having expressed — 
a vector — it is — although — this — rotation — a matrix -- R — ' — using — a degree — a type — like — 
coordinate transformation — carrying out — things — an image pick-up — the section — 1 1 — a 
photographic subject — a field — right — a pair — carrying out — having made — the time — a three 
dimension — a vector — p — 1 — changing — having . 
[0091] 

[Equation 24] 
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(25) 



Therefore, the image which amended gate distortion is obtained by making the x-coordinate and y- 
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coordinate of a formula (25) into the coordinate acquired after coordinate transformation. And the 
parameter which amends the gate distortion of a criteria image using a formula (25) is computable with 
the above procedures. 

[0092] Next, the coordinate transformation section 223 coordinate-transformation-creates a distortion 
amendment image for a criteria image based on the gate distortion amendment parameter computed by 
the parameter calculation section 222. The coordinate (xs, ys) before the conversion corresponding to 
the coordinate after coordinate transformation (X, Y) is calculated based on a formula (25), and, 
specifically, a interpolation operation determines the pixel value in a coordinate (X, Y) based on a pixel 
value [ / near the calculated coordinate (xs, ys) ]. In addition, what is necessary is just to perform this 
interpolation operation using the existing approaches, such as a congruence linear interpolation method 
and B-spline interpolation method. 

[0093] As mentioned above, when according to the image processing system concerning the gestalt 1 of 
operation of this invention a photographic subject side is photoed two or more sheets so that at least 
parts may overlap mutually, the gate distortion in the image of the photoed arbitration can be amended, 
and the overview of a more proper photographic subject can be obtained. 

[0094] That is, it is sufficient, if there are an image which photoed the whole photographic subject, and 
an image which photoed some photographic subjects as it is necessary to photo all the range of a 
photographic subject in no images in the above-mentioned procedure for example, and is shown in 
drawing 14 . And since the image which photoed the whole photographic subject by the criteria image 
setting section 20 can be chosen as a criteria image in such a case, the overview of the photographic 
subject with which gate distortion was amended is generable. 

[0095] Moreover, if the number of sheets to photo is limited to two sheets since there should just be a 
photographic subject image of two sheets at worst in order to generate a distortion amendment image by 
the above-mentioned approach, count cost which the input of an image and a setup of photography and a 
criteria image are not only simplified for a user, but detection of memory space, the focus, and 
corresponding points required in order to memorize an image, and count of a projective-transformation 
matrix take can be made small. 

[Gestalt 2 of operation] drawing 18 is drawing for explaining the image-processing approach and image 
processing system concerning the gestalt 2 of operation of this invention. As shown in drawing 18 (a), 
actuation of the image processing system 6 concerning the gestalt of this operation when the static image 
of the same photographic subject side PL is photoed from two or more directions dl-dk so that it may 
overlap in some images, respectively is explained. 

[0096] Here, as shown in drawing 18 (a), suppose that there is a duplication field, respectively by taking 
a photograph from Direction Dn by Hazama of Image imj and Image im (j+1) (1 <=j<=k -1) which were 
obtained by taking a photograph in the direction which Image imn (n= 1, 2 — , j, — , k) is obtained, for 
example, adjoins each other like an image iml and an image im2. 

[0097] And in the image processing system concerning the gestalt 2 of this operation, as shown in 
drawing 18 (b), other images are stuck and the synthetic image IMC is obtained so that it may have 
consistency in the gap or one image imj chosen as a criteria image. 

[0098] Drawing 19 is drawing showing the configuration of the image processing system 6 concerning 
the gestalt 2 of this operation. As shown in drawing 19 , although an image processing system 6 has the 
same configuration as the image processing system 1 concerning the gestalt 1 of the above-mentioned 
implementation shown in drawing 3 , it is different in that it has the image composition section 24 
instead of the image-distortion amendment section 22. Here, although the image composition section 24 
sticks an image with the field which overlaps mutually by carrying out coordinate transformation based 
on the relation of the focus and corresponding points which were obtained by the correspondence 
detecting element 21, it is later explained in detail about this actuation. 

[0099] Drawing 20 is a flow chart which shows actuation of the image processing system 6 concerning 
the gestalt 2 of this operation shown in drawing 19 . First, an electric power switch 101 is changed, an 
image processing system 1 is started, and photography mode is chosen. Here, the above-mentioned 
photography mode consists of the normal mode which takes the usual snapshot, and gate amendment 
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mode which generates the image which amended the gate distortion of the photoed image. And selection 
in this photography mode is made when a user operates the photography mode setting key 104. In 
addition, as the photography mode setting section 19, the photography mode setting key 104 is formed 
in the body of an image processing system 1 . However, the photography mode setting section 1 9 may 
consist of hardware or software etc. which are formed separately from a body. 
[0100] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 5 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0101] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And a user photos at least two photographic subject sides PL by the image pick-up section 1 1, 
and a photographic subject image is incorporated by the image processing system 6. In addition, each 
image needs to be photoed so that some photographic subject images may overlap mutually at this time. 
[0102] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. In addition, the above-mentioned 
directions are made by pressing the photography mode setting key 1 04 once again, and changing to the 
normal mode, and also they may form the switch for directing photography termination separately. 
[0103] And after photography of a photographic subject is completed as mentioned above, the actuation 
which generates the synthetic image which stuck the image of two or more sheets below on step S4 is 
started. In step S4, it chooses first on the basis of which image a synthetic image is generated. In 
addition, the image chosen at this time is the above-mentioned criteria image, and the case where the 
image imj shown in drawing 18 as an example is chosen as below as a criteria image is explained. 
Moreover, although a setup of this criteria image is made in the criteria image setting section 20, the 
configuration and actuation of the criteria image setting section 20 of it are the same as that of the case 
in the gestalt 1 of the above-mentioned implementation. 

[0104] Next, in step S5, the image pair photoed from the adjoining direction shown in drawing 18 (a), 
i.e., the corresponding points which show the same part as the focus and this focus between images n 
(n+1) (1 <=n<=k -1), is detected. Detection of this focus and corresponding points is performed by the 
correspondence detecting element 2 1 . Here, the configuration and actuation of the correspondence 
detecting element 21 are the same as that of the case in the gestalt 1 of the above-mentioned 
implementation. 

[0105] And termination of detection of the focus and corresponding points sticks an image, after 
carrying out coordinate transformation based on the relation of both the acquired points so that it may 
have consistency in a criteria image. Here, when using projective transformation as coordinate 
transformation, while calculating a projective-transformation matrix in step S6, a synthetic image is 
generated by the above-mentioned lamination in step S7. Such calculation of a projective-transformation 
matrix and generation of a synthetic image are performed by the image composition section 24. The 
configuration and actuation of this image composition section 24 are explained in detail below. 
[0106] Drawing 2 1 is drawing showing the configuration of the image composition section 24 shown in 
drawing 19 . As shown in drawing 2 1 , the image composition section 24 contains the projective- 
transformation calculation section 231 and the coordinate transformation section 232. Here, the 
projective-transformation calculation section 231 is connected to the criteria image setting section 20 
and the correspondence detecting element 21, and the coordinate transformation section 232 is 
connected to die criteria image setting section 20, a frame memory 15, and the projective-transformation 
calculation section 23 1 . 

[0107] And the projective-transformation calculation section 231 computes the projective- 
transformation matrix B shown by the formula (3) using the group of the four or more focus and 
corresponding points. The calculation procedure is the same as that of the case in the gestalt 1 of the 
above-mentioned implementation, and should just perform least square count of a formula (5). It is 
necessary to calculate the projective-transformation matrix between the image pairs of an individual (k- 
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1) by setting in the gestalt of this operation, and to search for the projective-transformation matrix from 
each image to a criteria image further here. 

[0108] As shown in drawing 18 , more specifically, they are Bn and a projective-transformation matrix 
from Image imn to Image imj about the projective-transformation matrix from Image imn (n= 1 to k-1) 
to Image im (n+1) [0109] 
[Equation 25] 

Bn— j 

If it sets, this projective-transformation matrix is calculable based on a degree type. 
[0110] 

[Equation 26] 

(26) 




<n> i) 



Here, the coordinate transformation section 232 is stuck [ Image / n ] in the image [ criteria ] imj based 
on the projective-transformation matrix shown by the formula (26) computed by the projective- 
transformation calculation section 23 1 . Specifically, the coordinate (x y) before the coordinate 
transformation in Image n corresponding to the coordinate after the coordinate transformation in a 
criteria image (X, Y) is first calculated based on a degree type. 
[0111] 

[Equation 27] 



y 

1 



Y 



(27) 



However, s is a constant for setting the third component of the column vector of the left part in a 
formula (27) to 1. Next, a interpolation operation determines the pixel value of a coordinate (X, Y) 
based on a pixel value [ / near the coordinate before coordinate transformation (x y) ]. And what is 
necessary is just to perform this interpolation operation using the existing approaches, such as a 
congruence linear interpolation method and B-spline interpolation method. 

[0112] As mentioned above, according to the image processing system 6 concerning the gestalt of this 
operation, the photographic subject side PL is photoed two or more sheets, and the synthetic image 
crossed to the large field of the photographic subject side PL is generated by sticking other images to the 
selected criteria image so that parts may overlap mutually. In this case, by the criteria image setting 
section 20, since the image photoed from the direction which carries out a right pair mostly to a 
photographic subject, i.e., the small image of gate distortion, can be chosen as a criteria image, it can 
instigate as a result and the small overall photographic subject image of distortion can be obtained as the 
above-mentioned synthetic image. 

After setting to the image processing system concerning the gestalten 1 and 2 of the [gestalt 3 of 
operation] above-mentioned implementation and inputting or photoing two or more photographic 
subject images previously, the criteria image which is made into the object of distortion amendment or is 
made into the criteria of a synthetic image is chosen. On the other hand, before the image processing 
system concerning the gestalt 3 of this operation photos a photographic subject, the above-mentioned 
criteria image is set up beforehand. 

[01 13] Drawing 22 is drawing showing the configuration of the image processing system 8 concerning 
the gestalt 3 of operation of this invention. As shown in drawing 22 , although the image processing 
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system 8 concerning the gestalt of this operation has the same configuration as the image processing 
system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at the point further 
equipped with the notice section 26. Here, the notice section 26 is connected to the main control section 
14 and the criteria image setting section 20. 

[0114] In addition, the shutter 113 and interface 16 which were included in this image processing system 
8 are connected to the main control section 14, and a finder 241 is connected to an interface 16. 
[0115] Next, actuation of the image processing system 8 concerning the gestalt 3 of operation of this 
invention is explained, referring to the flow chart of drawing 23 . First, if it is judged whether it 
instigates in step S 1 and amendment mode is chosen, and gate amendment mode is not chosen but the 
normal mode is chosen, it will progress to step S10 and the snapshot of a desired photographic subject 
will be taken by the user. 

[0116] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And in step S2, the image made applicable [ of gate distortion ] to amendment, i.e., a criteria 
image, is set as the criteria image setting section 20. The criteria image setting section 20 contains a 
cursor key 204 and the decision key 203 here, as shown in drawing 24 . And while an overlay indication 
of the alphabetic character "criteria image setup" is given at a display 17, it points so that the number of 
sheets photoed to a user after this and the criteria image assignment value which specifies whether the 
image photographed to the how many sheets in this photography is used as a criteria image may be set 
up. Here, a user can set up desired photography number of sheets and a criteria image assignment value 
by making the set point within the limit fluctuate by operating a cursor key 204 top or a down scrolling 
key by changing between setup of a criteria image to photography number of sheets, and operating the 
left or a rightward scrolling key. In addition, a setup of the above-mentioned criteria image is completed 
by pressing the decision key 203. 

[0117] Next, in step S3, a user starts photography of the photographic subject image of at least two or 
more sheets. In addition, at this time, each image needs to be photoed so that parts may overlap 
mutually. And by supplying a photography signal to the main control section 14 from a shutter 113, 
whenever it photos an image, the increment of the counter built in the main control section 14 is carried 
out, and the number-of-sheets specification signal which shows the how many sheets the next 
photography is from this counter is supplied to the notice section 26. Since the signal which shows the 
above-mentioned criteria image assignment value from the criteria image setting section 20 is supplied 
to the register built in the notice section 26 here and the criteria image assignment value is stored in this 
register, In step S4, the notice section 26 compares the above-mentioned criteria image assignment value 
always stored in this register with the value which the above-mentioned number-of-sheets specification 
signal shows, and judges whether the image photoed shortly is what is used as a criteria image. 
[01 18] And in the above-mentioned comparison, both value is in agreement, when the image which it is 
going to photo from now on is what is used as a criteria image, it progresses to step S5, and the notice 
section 26 notifies a user of it being photography of a criteria image. That is, at this time, from the notice 
section 26, a notice signal is supplied to an interface 16 through the main control section 14, and as 
shown in drawing 25 , according to this notice signal, the indicator 242 of photographic subject image 
35 width in a finder 241 is turned on with an interface 16. Therefore, it is recognized by the user at the 
time of photography whether it is photography of a criteria image easily. In addition, the notice in the 
above may be performed by displaying a predetermined text and a predetermined symbol on a display 
17 etc. 

[01 19] Next, although it progresses to step S6, when the image photoed next in step S4 is judged not to 
consider as a criteria image, it progresses to the direct step S6. And in this step S6, when it is judged 
whether photography (input of a photographic subject image) of a photographic subject was completed 
and it is judged that it does not end, it returns to step S3. On the other hand, when it is judged that it 
ends, it progresses to step S7. Here, it shall be made by pushing the switch with which a setup [ in / 
termination of photography is judged according to directions of a user, and / in these directions / the 
photography mode setting section 19] was changed to the normal mode, or was prepared for 
photography termination etc. 
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[0120] Next, at step S7, the focus is detected in a criteria image and the corresponding points which 
show the same part as the above-mentioned focus in the image which overlaps in these some criteria 
images [ at least ] are detected. Detection of such the focus and corresponding points is performed by the 
correspondence detecting element 2 1 shown in drawing 22 . In addition, the configuration and actuation 
of the this detecting element 21 are the same as that of the case of the gestalt 1 of the above-mentioned 
implementation. 

[0121] And while computing the parameter for amending the gate distortion of a criteria image in step 
S8, the image which instigated based on the above-mentioned parameter in step S9, and amended 
distortion is generated, and actuation is ended. In addition, although generation of the image which 
amended calculation of this parameter and gate distortion is performed by the image-distortion 
amendment section 22, the configuration and actuation of this image-distortion amendment section 22 
are the same as that of the case of the gestalt 1 of the above-mentioned implementation. 
[0122] As mentioned above, since a user is notified by the notice section 26 in the image photography 
by the user by setting up a criteria image assignment value before photography of whether the image 
photoed next is what is used as a criteria image according to the image processing system 8 concerning 
the gestalt 3 of this operation, a user can recognize photography of a criteria image easily. And a user 
can pay [ that the field of a request of a photographic subject especially goes into photographic 
coverage, and ] attention at the time of photography of a criteria image. Furthermore, the photography 
mistake of a criteria image can be decreased. 

[0123] In addition, it cannot be overemphasized that the technique which notifies a user of photography 
of a criteria image according to a setup of a criteria image assignment value by the notice section 26 can 
be applied also to the image processing system 6 concerning the gestalt 2 of the above-mentioned 
implementation. 

Although the image processing system concerning the gestalt of the [gestalt 4 of operation] above- 
mentioned implementation needed to photo the photographic subject twice [ at least ] from a different 
direction by moving the single optical system included in the image pick-up section, with the image 
processing system concerning the gestalt of this operation, two or more optical system is installed in the 
image pick-up section 41 side by side, and two or more sheets are once obtained by photography in the 
photographic subject image photoed from a different direction. 

[0124] Drawing 26 is drawing showing the configuration of the image processing system 40 concerning 
the gestalt 4 of operation of this invention. As shown in drawing 26 , although the image processing 
system 40 concerning the gestalt 4 of this operation has the same configuration as the image processing 
system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different from the image pick- 
up section 41 at a point including two optical system 1 1 A and 1 IB. 

[0125] Actuation of the image processing system applied to the gestalt of this operation below is 
explained referring to the flow chart of drawing 27 . First, an image processing system 40 is started and 
photography mode is chosen. Here, the above-mentioned photography mode consists of the normal 
mode which takes the usual snapshot, and gate amendment mode which generates the image which 
amended the gate distortion of the photoed image. And selection in this photography mode is made 
when a user operates a photography mode setting key. In addition, as the photography mode setting 
section 19, the photography mode setting key 104 is formed in the body of an image processing system 
40. However, the photography mode setting section 19 may consist of hardware or software etc. which 
are formed separately from a body. 

[0126] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 27 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0127] The criteria image as an object which instigates in step SI, will progress to step S2 if amendment 
mode is chosen, instigates to the criteria image setting section 20 on the other hand, and amends 
distortion is set up. As shown in drawing 28 , the above scrolling key 201 and the down scrolling key 
202, and the decision key 203 are contained in the criteria image setting section 20 here. Moreover, a 
user is asked for selection whether to use as the above-mentioned criteria image the image photoed by 
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which optical system 1 1 A and 1 IB while an overlay indication of the alphabetic character "criteria 
image setup" is given at a display 17. 

[0128] Then, by operating the above scrolling key 201 or the down scrolling key 202, a user moves the 
pointer shown with a triangle on a display 17, and specifies either the display the "camera 1" which 
chooses optical-system 1 1 A, or the display the "camera 2" which chooses optical-system 1 IB. And if 
the above-mentioned pointer presses the decision key 203 where one of optical system is specified, let 
the image photoed by the this specified optical system be the above-mentioned criteria image. Here, the 
setting information on the above-mentioned criteria image is supplied to the main control section 14 
from the criteria image setting section 20. 

[0129] Next, in step S3, a user photos a photographic subject. At this time, it is easily checked 
[ displaying only the image photoed by the optical system chosen at step S2 by control by the main 
control section 14, then ] by the user at a display 17 whether the image which is going to amend gate 
distortion includes the range of the request of a photographic subject. 

[0130] And in step S4, while the focus is detected within a criteria image, the corresponding points 
which show the same part as this focus within the image which has the field which overlapped the 
criteria image are detected. In addition, although detection of this focus and corresponding points is 
performed by the correspondence detecting element 21, the configuration and actuation of this 
correspondence detecting element 2 1 are the same as that of the case of the gestalt 1 of the above- 
mentioned implementation. 

[0131] Next, while computing the parameter for amending the gate distortion of a criteria image in step 
S5, the image which instigated based on this parameter in step S6, and amended distortion is generated, 
and actuation is ended. Although generation of the image which amended calculation of the above- 
mentioned parameter and gate distortion is performed by the image-distortion amendment section 22 
here, the configuration and actuation of this image-distortion amendment section 22 are the same as that 
of the case of the gestalt 1 of the above-mentioned implementation. 

[0132] As mentioned above, according to the image processing system 40 concerning the gestalt 4 of 
this operation, since the image pick-up section 41 is equipped with at least two optical system, a user 
can get the amendment image obtained with the image processing system applied to the gestalt of the 
above-mentioned implementation by one photography actuation. Moreover, if only the image photoed in 
the selected optical system is displayed on a display 17, since whether the image which is going to 
amend gate distortion includes the range of the request of a photographic subject can check easily by the 
user, a user can pay attention further to photography of a criteria image, and can decrease the possibility 
of a photography mistake. 

[0133] Moreover, it cannot be overemphasized that the image processing system 6 concerning the 
gestalt 2 of the above-mentioned implementation may be equipped with the image pick-up section 
which three or more optical system may be included in the image pick-up section 41 shown in drawing 
26 , and includes such two or more optical system. 

[0134] In addition, in the image processing system concerning the gestalt of implementation of all 
above, instead of photoing a photographic subject by the image pick-up section, the photographic 
subject image of two or more sheets stored in storages, such as storage, such as a hard disk, and CD- 
ROM, may be incorporated in enternal memory section 1 8 grade, and an amendment image may be 
generated using these photographic subject images. Moreover, that by which the photography mode 
setting section 19, the criteria image setting section 20, the correspondence detecting element 21, the 
image-distortion amendment section 22, and image composition section 24 grade are stored in case with 
the another image pick-up sections 1 1 and 41, for example, a computer etc., is considered to be the 
gestalt of the above-mentioned implementation the same way. 

[0135] Moreover, although the correspondence detecting element 21 explained that corresponding points 
were detected by concentration matching by the correlation technique, you may carry out by another 
technique, such as a differentiation between space-time. Furthermore, in calculation of a gate distortion 
parameter, it is not restricted to the coordinate transformation shown in the above formulas (20), but 
other parameter calculation approaches may be applied. 
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In the gestalt of the [gestalt 5 of operation] above-mentioned implementation, although the coordinate 
transformation parameter was computed to the criteria image made into the object of an image 
processing and the distortion amendment image was generated using the interpolation operation, as for 
this criteria image, it is desirable that it is the image photoed by an image with little degradation of the 
image by the interpolation operation, i.e., the condition of having carried out the right pair to the 
photographic subject comparatively, including abundant paintings-and-calligraphic-works information. 
Then, the image processing system which performs the image-processing approach which chooses the 
optimal criteria image as below automatically, and this approach is explained. 
[0136] Drawing 29 is drawing showing the first example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 29 , although the 
image processing system 50 concerning the gestalt 5 of this operation has the same configuration as the 
image processing system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at 
the point equipped with the photographic subject field judging section 25 and the criteria image 
automatic selection section 27 instead of the criteria image setting section 20. 

[0137] Here, the photographic subject field judging section 25 and the criteria image automatic selection 
section 27 are connected to the main control section 14, respectively, and the outgoing end of the 
photographic subject field judging section 25 is connected to the criteria image automatic selection 
section 27. Moreover, the outgoing end of the criteria image automatic selection section 27 is connected 
to the image-distortion amendment section 22. 

[0138] In addition, also in the image processing system 50 concerning the gestalt 5 of this operation, as 
shown in drawing 4 , it can consider as the same configuration as the image processing system 1 
concerning the gestalt 1 of the above-mentioned implementation. 

[0139] It explains referring to the flow chart shown in drawing 30 in actuation of the image processing 
system 50 applied to the gestalt 5 of this operation below. The image processing system 50 applied to 
the gestalt 5 of this operation here is explained in detail focusing on difference below in order to operate 
like the image processing system 1 concerning the gestalt 1 of the above-mentioned implementation. 
[0140] First, in step SI, when it is judged whether gate amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen, it progresses to step S10, and a desired 
snapshot is taken by the user by this user. 

[0141] On the other hand, when it instigates by the user in step SI and amendment mode is chosen, it 
progresses to step S2. And in step S2, the photographic subject image two-times photography was 
carried out [ the image ] at least by the image pick-up section 1 1 is incorporated to a frame memory 15. 
In addition, at this time, each image needs to be photoed so that these some photographic subject images 
may overlap mutually. 

[0142] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. 

[0143] And in step S4, the focus is detected by the correspondence detecting element 21 in a 
photographic subject image, and the corresponding points which show the same part as the above- 
mentioned focus in the reference image which overlaps in these some photographic subject images [ at 
least ] are detected. Next, after detection of the focus and corresponding points is completed, the 
actuation which instigates below and amends distortion is started. And in step S5, the criteria image 
automatic selection section 27 makes first automatic selection of the criteria image which is instigated 
and is made into the object of distortion amendment. The criteria image automatic selection section 27 is 
explained in detail below. 

[0144] In addition, the parameter for instigating in step S6 to the criteria image chosen in step S5 like 
the image processing system 1 concerning the gestalt 1 of the above-mentioned implementation, and 
amending distortion is calculated, the image which amended the gate distortion of an image based on 
this parameter in step S7 is created, and actuation is ended. 

[0145] When amending gate distortion, it is suitable, if it crossed to the field where a photographic 
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subject side is the largest in two or more photoed images, and is reflected as the above-mentioned 
criteria image and an image with the abundant paintings-and-calligraphic-works amount of information 
which a user needs is chosen. Moreover, it is suitable if the image with the smallest possible angle (it is 
also called a "gate angle" to below) made to the photographic subject side of the image pick-up side at 
the time of photography is chosen. The reason is explained referring to drawing 3 1 . 
[0146] Although the image-distortion amendment section 22 shown in drawing 29 is instigated by 
performing coordinate transformation by the formula (25) as mentioned above and distortion 
amendment is performed, this amendment actuation changes according to the gate angle phi. Here, in 
order to give explanation an easy thing, the gate angle phi is restricted to the circumference of the y-axis, 
and it is assumed that the magnitude of the plane of projection 31-ed parallel to a photographic subject 
side is equal to the magnitude of the image pick-up side 32. 

[0147] As shown in drawing 3 1 (a), in the this [ angle / phi / gate ] coordinate transformation in the case 
of being comparatively small, the point near the left end of the image pick-up side 32 is projected by the 
variations shown by the vector which goes to Zero o to the plane of projection 31-ed. In addition, in the 
slash section in the plane of projection 3 1-ed, the resolution of a photographic subject image falls by 
such coordinate transformation. 

[0148] On the other hand, as shown in drawing 3 1 (b), it instigates in this coordinate transformation, and 
the point near the left end of the image pick-up side 32 is projected on the plane of projection 3 1-ed with 
a bigger variations than the variations angle phi is indicated to be to drawing 3 1 (a) when comparatively 
large so that the position vector to Zero o may be reduced more greatly. And it turns out that the field to 
which the resolution of a photographic subject image falls by this coordinate transformation becomes 
large as compared with the case of drawing 3 1 (a). 

[0 1 49] It turns out that degradation of the resolution which followed, as mentioned above originated in 
the coordinate transformation by the image-distortion amendment section 22, so that the gate angle phi 
was small decreases. 

[0150] Moreover, although the photographic subject field judging section 25 shown in drawing 29 
detects the field which a photographic subject occupies in the photoed image For example, as indicated 
by reference "processing of an image and recognition" (Takeshi Akoin and Tomoharu Nagao 
collaboration, Shokodo) (a) How to cluster on an image like a field grown method or a field split plot 
experiment, (b) The field division approaches using the edge in images, such as the approach of 
clustering on feature spaces, such as field division by the histogram, and (c) profile line tracking, such as 
an approach and the (d) texture analysis, are applied. And the photographic subject field judging section 
25 chooses the image which photoed the large range of a photographic subject according to this 
judgment result as the above-mentioned criteria image. Consequently, an image including paintings-and- 
calligraphic-works information abundant as a criteria image can be chosen automatically. 
[0151] The image processing system applied to the gestalt 5 of operation of this invention here can also 
be considered as a configuration as shown in drawing 32 . That is, although the image processing system 
51 shown in drawing 32 has the same configuration as the image processing system 50 shown in 
drawing 29 , it is different at the point equipped with the straight-line-like pattern detecting element 28 
instead of the photographic subject field judging section 25. 

[0152] Generally, in the photographic subject side which makes a document the start, many straight- 
lines [, such as a character string and a ruled line, / which have an parallel relation mutually ]-like 
patterns exist. However, when the gate angle at the time of photography is large, it is projected on an 
image as a straight-line-like pattern in which the straight-line-like pattern which must originally be 
parallel has different sense. Therefore, by investigating dispersion in the sense of the straight-line-like 
pattern projected on the image, the size of this gate angle can be distinguished and an image with a small 
gate angle can be automatically chosen as a criteria image. 

[0153] Here, the straight- line-like pattern detecting element 28 shown in drawing 32 performs 
processing which detects a straight-line-like pattern in two or more photoed images. And an example of 
the detection approach of this straight-line-like pattern is explained below. 

[0154] First, an edge image is created by taking differential in these two or more images. Next, the 
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fragmentary edge point group in the created edge image is divided into a straight-line-like segment, and 
each straight-line-like segment is applied to the straight-line equation shown in the following equations 
(28). 

ax+by+c=0 (a2+b2=l) (28) 

here — the above-mentioned formula (28) — applying — it is made by applying a least square method 
using the point group which constitutes a straight-line-like segment. And termination of the reliance 
panel of the straight line to a straight-line-like segment searches for dispersion in being a parameter 
showing the linear sense (a, b). Thus, dispersion in the sense of the straight line in each image can be 
known by searching for dispersion in the parameter to two or more above-mentioned images of all (a, b). 
And an image with the smallest dispersion in the linear sense can be automatically chosen as a criteria 
image. 

[0155] Moreover, in automatic selection of a criteria image, the following approaches are also 
applicable. First, in two or more above-mentioned images, an edge image is created by the same 
approach as the above. Next, Hough conversion is performed to all the points in this edge image. Hough 
conversion is mathematical conversion used in order to detect a straight line from a fragmentary edge, 
and is the space constituted with the parameter of the formula expressing a line to detect here, and the 
approach of clustering is said. And more specifically, the sequence of points on an image are projected 
to the theta-rho space shown by the x axis, and angle theta and linear die-length rho to make. 
[0156] Drawing 33 is drawing explaining Hough conversion to the theta-rho space shown in drawing 33 
(b) from the image space shown in drawing 33 (a). Each point P1-P3 on an image is changed into 
drawing 33 by Hough conversion at the curves L1-L3 which correspond, respectively so that it may be 
shown. And although the point CP which the locus of this curve is concentrating in theta-rho space will 
arise if such conversion is performed about the all points on an image, this point CP is equivalent to the 
straight line which passes along many edge points in an edge image. If the coordinate of this point CP is 
set to (theta, rho), it will mean that the straight line corresponding to the following formula (29) was 
detected here. 

Rho==xcostheta+ysintheta (29) 

And dispersion in the sense of the straight line in each image can be known by extracting many points 
CP which this curve concentrates to two or more above-mentioned images of all, and searching for 
dispersion in theta in these points. Thus, what is necessary is just to choose automatically an image with 
the smallest dispersion in the linear sense as a criteria image. 

[0157] Drawing 34 is drawing showing the third example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 34 , the image 
processing system concerning the gestalt 5 of this operation is good also as connecting the criteria image 
automatic selection section 27 to the correspondence detecting element 21, without having the 
photographic subject field judging section 25 shown in drawing 29 , and the straight-line-like pattern 
detecting element 28 shown in drawing 32 . 

[0158] The group of the focus detected by the detecting element 21 corresponding to the above and 
corresponding points is used for count of the parameter which amends the projective-transformation 
matrix B, i.e., gate distortion. At this time, generally, and it is computed with so sufficient that they are 
distributing broadly a precision. [ this parameter ] [ the group of the above-mentioned focus and 
corresponding points ] Then, the distribution in the groups of the focus detected by the correspondence 
detecting element 21 and corresponding points and those images may be investigated to two or more 
photoed images, and the image those values of whose are maxes may be automatically chosen as a 
criteria image. 

[0159] Moreover, in the correspondence detection based on the above-mentioned correlation technique, 
much focus and corresponding points are detected, so that there are many patterns which generally have 
the description in an image. And since possibility that many paintings-and-calligraphic-works 
information useful for a user in an image is included is high, if the focus and corresponding points make 
automatic selection of the abundant images as a criteria image, that there are many characteristic 
patterns can instigate an image including required paintings-and-calligraphic-works information, it can 
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set it as the object of distortion amendment, and is suitable. 

[0160] Drawing 35 is drawing showing the fourth example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 35 , the image 
processing system concerning the gestalt 5 of this operation is good also as having the flat-surface 
measurement section 29 instead of the photographic subject field judging section 25 shown in drawing 
29 , and the straight-line-like pattern detecting element 28 shown in drawing 32 . 
[0161] Here, the flat-surface measurement section 29 measures the sense of the photographic subject 
over the image pick-up section 1 1 at the time of photography of each image. Drawing 36 is drawing 
showing the example of a configuration of the flat-surface measurement section 29 shown in drawing 
35 . As shown in drawing 36 , the flat-surface measurement section 29 is equipped with the spot light 
source 271, a photo detector 272, the three-dimension coordinate calculation section 273, and the flat- 
surface calculation section 274, and the spot light source 271 contains light source 271a which consists 
of light emitting diode, semiconductor laser, etc., scan mirror 271b, such as a polygon mirror, and 
mechanical-component 271c which controls a motion of scan mirror 271b. 
[0162] Here, in the above-mentioned spot light source 271, scan mirror 271b is controlled by 
mechanical-component 271c so that the spot light generated by light source 271a hits the photographic 
subject side PL. Moreover, a photo detector 272 is constituted by optoelectric transducers by which the 
location to the spot light source 271 was installed in the location measured beforehand, such as PSD 
(Position sensitive detector) and CCD, and detects the sense of the reflected light from the photographic 
subject side PL. In addition, the optoelectric transducer 114 contained in the image pick-up section 1 1 
may be used as a photo detector 272 in the above. 

[0163] Moreover, the three-dimension coordinate calculation section 273 computes the three-dimension 
coordinate (X, Y, Z) of the photographic subject side PL on the basis of an image processing system 53 
by using the principle of triangulation according to the sense of spot light which the spot light source 
271 irradiated, and the physical relationship of the spot light source 271 and a photo detector 272 and 
the sense of the reflected light which the photo detector 272 detected. And the flat-surface calculation 
section 274 presumes a flat-surface equation using the three-dimension coordinate of three or more 
points which is not on the same straight line computed by the three-dimension coordinate calculation 
section 273. For example, it is the flat-surface equation for which it asks aX+bY+cZ+d=0 (a2+b2+c2=l, 
c> 0) (30) 

It sets and four parameters (a, b, c, d) in the above-mentioned flat-surface equation are calculated with a 
least square method using the three-dimension coordinate of three or more points. Consequently, the 
above-mentioned gate angle phi is calculated by the degree type (31). 
Phi=cos- lc(31) 

Therefore, what is necessary is to measure the sense of a photographic subject side to two or more 
photoed images of all, and just to choose automatically the image the gate angle phi of whose is min as a 
criteria image. 

[0164] In addition, as described in explanation of the image-distortion amendment section 22, it can ask 
for the sense of this photographic subject also by using the group of the focus and corresponding points 
which were detected by the correspondence detecting element 2 1 . Therefore, the image processing 
system concerning the gestalt 5 of this operation can once perform a formula (14) thru/or a formula (18) 
in the image-distortion amendment section 22, can ask for the sense of a photographic subject, and can 
also consider the obtained result as the configuration of outputting to the criteria image automatic 
selection section 27. 

As shown below in [the gestalt 6 of operation] at drawing 18 , the gestalt of the operation on condition 
of the case where the static image in the same photographic subject side PL is photoed from two or more 
directions so that it may overlap in some images, respectively is explained. In addition, as shown in 
drawing 18 here, by Hazama of Image imj and Image im (j+l) (1 <=j<=K -1) which were obtained by 
taking a photograph in the direction which Image imj (1 <=^<=K) is obtained, for example, adjoins each 
other like an image iml and an image im2, there shall be a duplication field by taking a photograph from 
Direction dj, respectively. And in the image processing system concerning the gestalt 6 of this operation, 
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other images are stuck and a synthetic image is generated so that it may have consistency in the gap or 
one image chosen as a criteria image. 

[0165] Drawing 37 is drawing showing the configuration of the image processing system 60 concerning 
the gestalt 6 of operation of this invention. As shown in drawing 37 , although the image processing 
system 60 concerning the gestalt 6 of this operation has the same configuration as the image processing 
system 6 concerning the gestalt 2 of operation shown in drawing 19 , it is different at the point equipped 
with the criteria image automatic selection section 27 instead of the criteria image setting section 20. 
[0166] The image processing system 60 concerning the gestalt 6 of this operation which has the above 
configurations is explained focusing on difference, referring to the flow chart shown in drawing 3 8 in 
actuation of the image processing system 60 applied to the gestalt 6 of this operation below, although it 
operates like the image processing system 6 concerning the gestalt 2 of operation. 
[0167] First, in step SI, when it is judged whether image composition mode is chosen, and image 
composition mode is not chosen but the normal mode is chosen, it progresses to step S10, and a desired 
snapshot is taken by the user by this user. 

[0168] On the other hand, when image composition mode is chosen by the user in step SI, it progresses 
to step S2. And in step S2, two or more photographic subject images photoed by the image pick-up 
section 1 1 are incorporated to a frame memory 15. In addition, at this time, each image needs to be 
photoed so that these some photographic subject images may overlap mutually. 

[0169] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. 

[0170] And in step S4, the focus is detected by the correspondence detecting element 21 between Image 
imj and Image im (j+1 ), and the corresponding points which show the same part as the above-mentioned 
focus in the image im (j+1) which overlaps in these some images [ at least ] imj are detected. 
[0171] Next, after detection of this focus and these corresponding points is completed, the actuation 
which generates the synthetic image which stuck the image of two or more sheets on below is started. 
And in step S5, the criteria image automatic selection section 27 chooses automatically first the criteria 
image made into criteria in this composition. Here, the criteria image automatic selection section 27 
chooses automatically the image with the above-mentioned smallest possible gate angle as a criteria 
image. And the synthetic small image of gate distortion can be obtained by performing such selection. 
[0172] And in step S6, the projective-transformation matrix for performing composition on the basis of 
the criteria image chosen in step S5 is computed, a synthetic image is generated using this projective- 
transformation matrix in step S7, and actuation is ended. 

[Gestalt 7 of operation] drawing 39 is drawing showing the configuration of the image processing 
system concerning the gestalt 7 of operation of this invention. As shown in drawing 39 , although the 
image processing system 70 concerning the gestalt 7 of this operation has the same configuration as the 
image processing system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at 
the point further equipped with the criteria image automatic selection section 27 and the change section 
45. 

[0173] Here, the criteria image automatic selection section 27 is controlled by the main control section 
14. Moreover, as for the change section 45, an outgoing end is connected to the image-distortion 
amendment section 22 while the input edge is connected to the criteria image setting section 20 and the 
criteria image automatic selection section 27. 

[0174] Moreover, the change section 45 changes the selection approach of a criteria image, and as 
shown in drawing 40 , it contains the above scrolling key 201, the down scrolling key 202, and the 
decision key 203. Here, when a user pushes alternatively the menu screen key (not shown) displayed on 
a display 17, the screen where an overlay indication of the alphabetic character "selection of a criteria 
image" as shown in drawing 40 was given is displayed on a display 17. 

[0175] And when the above scrolling key 201 or the down scrolling key 202 is operated by this user, the 
cursor of the triangle shown in drawing 40 carries out vertical migration. At this time, if the decision key 
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203 is pressed in the condition that this cursor has pointed out the alphabetic character of "AUTO", the 
criteria image automatic selection section 27 as which choosing a criteria image automatically was 
determined and it was indicated to be to drawing 39 will be activated alternatively. On the other hand, if 
the decision key 203 is pressed in the condition that this cursor has pointed out the alphabetic character 
of "MANUAL", the criteria image setting section 20 as which setting up a criteria image with hand 
control was determined, and it was indicated to be to drawing 39 will be activated alternatively. 
[0176] Therefore, according to the image processing system concerning the gestalt 7 of this operation, 
by the change by the change section 45, since either the criteria image setting section 20 or the criteria 
image automatic selection section 27 is alternatively connectable with the image-distortion amendment 
section 22, a user can choose automatic or manual either as arbitration as the selection approach of a 
criteria image. 

[0177] In addition, also in the gestalt of which operation mentioned above, the image-processing 
approach concerning the gestalt of this operation can be described as a computer program. And as 
shown in drawing 4 1 , the above-mentioned image processing is easily realizable by equipping an image 
processing system 1 with the record medium 301 which stored this program, and making an image 
processing system 1 perform this program. 

[0178] Moreover, as shown in drawing 42 , the above-mentioned image processing is realizable also by 
equipping a personal computer (personal computer) PC with CD-ROM302 which stored this program, 
and performing this program with a personal computer PC. In addition, as a record medium with which 
a personal computer PC is equipped and which stores this program, it is not restricted to above- 
mentioned CD-ROM302, for example, it cannot be overemphasized that you may be DVD-ROM etc. 
[0179] An example in case this program execution realizes the above-mentioned image-processing 
approach below is explained. In this case, an image processing system 1 and a personal computer PC are 
incorporated to the above-mentioned signal-processing section 12 through the various interfaces in 
which the photographic subject image of two or more sheets stored in record media, such as storage, 
such as built-in memory and a hard disk, and CD-ROM, was carried by the computer. 
[0180] Moreover, a setup in the photography mode in the above is carried out by carrying out the mouse 
click of the icon displayed on push or a screen in the predetermined key in the keyboard of a personal 
computer PC. The image by which infanticide was carried out [ above-mentioned ] on the other hand by 
operating on a curtailed schedule and displaying two or more inputted images on a screen, pushing the 
vertical cursor key of the keyboard used for a calculating machine, or clicking on an icon with a mouse 
in choosing a criteria image with hand control etc. is chosen. And if a line feed key is pressed where a 
desired image is chosen, this image will be set up as a criteria image. 

[0181] Moreover, about the focal distance f of the optical system which constitutes the image pick-up 
section 1 1 , the focal distance of this optical system is measured beforehand, and it records on the interior 
of this record medium. And a user chooses the focal distance of the optical system used on the screen on 
the occasion of actual photography. Here, this focal distance is also recordable as header information. 
That is, when using an Exif format, for example as image data, the focal distance at the time of 
photography can be recorded as the header information. And the image processing system concerning 
the gestalt of this operation can obtain this focal distance by reading this header information. In addition, 
when the above approaches cannot be taken, it is good to display a dialog box etc. on screens, such as a 
personal computer PC, and to make a user do the manual entry of the direct focal distance into this 
dialog box. 

[0182] Moreover, it can treat about direction of the photographic subject in the above as well as the 
above-mentioned focal distance. That is, the sense of the photographic subject measured by the flat- 
surface measurement section 29 or the image-distortion amendment section 22 is beforehand recorded as 
header information in an image data file, and the image processing system concerning the gestalt of this 
operation is good also as obtaining the sense of this photographic subject by reading this header 
information. 

[0183] Although the field for recording the sense of a photographic subject does not exist in using an 
Exif format as this time, for example, image data, the sense of a photographic subject is recordable on 
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the field which the manufacturer called Maker Note can use freely. 

[0184] Moreover, when the above approaches cannot be taken, it is good to display a dialog box etc. on 
screens, such as a personal computer PC, and to make it make a user input the sense of a direct 
photographic subject like the case of a focal distance. 

[0185] In addition, the gestalt of operation of above-mentioned this invention can be applied to image 
processings using a digital still camera or a digital camcorder, such as a space information input and 
lamination composition of a division image, and can be applied also to the image device of a non- 
contact handy scanner or others. 

[Effect of the Invention] Since the image optimal in order to obtain a proper image by choosing the 
object which amends distortion out of two or more images in the image-processing approach which 
amends distortion of the image photoed from two or more directions can be made applicable to 
amendment like **** according to this invention so that at least a part may overlap to a photographic 
subject, this more accurate amendment is realizable. 

[0186] Moreover, distortion chooses fewest images from two or more images, and if the image which 
amended distortion is compounded with the this chosen image, a more proper synthetic image can be 
obtained. 

[0187] Here, if the object of amendment is automatically chosen according to the size of the field which 
a photographic subject occupies into an image, since the paintings-and-calligraphic-works amount of 
information needed can amend distortion of the most abundant images automatically, a certainly proper 
image can be obtained. 

[0188] Moreover, if the object of amendment is automatically chosen according to the sense of the 
straight-line- like pattern detected in an image, since the image photoed from the location which carried 
out the right pair to the photographic subject mostly can be automatically made applicable to 
amendment, a proper image with high resolution can be obtained. 

[0189] Moreover, if the object of amendment is automatically chosen according to the specified 
correspondence relation, since high amendment of precision can be performed certainly, the 
dependability of an image processing can be raised. 

[0190] Moreover, if the object of amendment is automatically chosen according to the sense of the 
photographic subject detected for every photography, since the image photoed from the location which 
carried out the right pair mostly to the photographic subject can be automatically made applicable to 
amendment, a proper image with high resolution can be obtained. 

[0191] Moreover, since according to the image processing system equipped with a notice means to 
notify a user of becoming the image made into the criteria at the time of the image photoed next 
amending distortion attention is called in the photography of an image made into criteria to a user in 
case distortion is amended, the photography mistake by a user's inattention etc. is avoided and 
dependability of operation and the quality of an amendment image can be raised. 
[0192] Moreover, according to the image processing system equipped with two or more optical means 
which photo a photographic subject to coincidence, and a selection means to choose the image made 
into the object of amendment among the images photoed by two or more optical means Since two or 
more photographic subject images once photoed by photography from two or more directions can be 
obtained, while actuation is made simple by decreasing the count of photography needed in order to 
amend distortion, the image with which distortion amendment was carried out more simply can be 
obtained. 



[Translation done.] 
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* NOTICES * 

JPO and NCI PI are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the record medium in which computer reading for 
realizing the image-processing approach, the image processing system, and this image-processing 
approach for not being based on a photography condition but obtaining a proper image in more detail 
about the image-processing approach, an image processing system, and a record medium is possible. 
[0002] 

[Description of the Prior Art] It will be necessary to also diversify the state of business with the rapid 
advance of a computer network, and to acquire important information quickly on all aspects of affairs. In 
connection with it, the demand to inputting goods required for business and document information 
simple and with high definition everywhere, making full use of the input device of a pocket mold is 
increasing. Especially, the application that it will utilize as electronic intelligence effective in work or 
amusement also came to be seen by performing processing and processing to the photoed image with 
rapid spread and its raise in resolving of a digital still camera. 

[0003] Although the distortion of an image (this is also called "gate distortion" .) arises as the first typical 
application when photoing photographic subject sides, such as A4 space and a large-sized poster, and an 
image pick-up side and a photographic subject side take a photograph in the condition which is not 
parallel, there is a technique of raising the readability of the document image information acquired by 
amending this. Here, in the image which wants to amend "gate distortion", it is required that the range of 
the request of a photographic subject should be copied. That is, it is necessary to photo at least one in the 
image of two or more sheets which the user photoed so that the range of the request of a photographic 
subject may be included. Therefore, the interface (I/F) which can choose the object which amends gate 
distortion among the photoed images of two or more sheets, and an interface to which suitable cautions 
are urged to a user at the time of photography are desired. 

[0004] Moreover, there are some which create the synthetic image of one sheet by carrying out division 
photography of the pattern drawn on the space information and the panels of a large area, such as a 
newspaper, or a wall by the portable picture input device as the second application, and sticking the 
obtained image of two or more sheets. That is, although the resolution of a digital camera is improving 
in recent years with the increment in the number of pixels of the image sensor represented by CCD 
(Charge Coupled Device), photoing the photographic subject which has a pattern fine as mentioned 
above, i.e., the photographic subject containing a high frequency component, and electronizing runs 
short of resolution still more. Therefore, by sticking an image, a highly minute image is created in false 
and the approach of compensating the lack of resolution of a digital camera is made. 
[0005] When a photographic subject can regard it as a plane, a geometric correction type like affine 
transformation or projective transformation is used, it is the technique which sticks the image which 
carried out division photography of some photographic subjects, and such an application changes the 
photographic subject image of each division image into how the photographic subject in the image used 
as criteria appears, and sticks. In addition, the outline of such a technique is indicated by reference 
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"computer vision-technical criticism and future view (new technical communique SHONZU besides 
Takashi Matsuyama). 

[0006] However, in such an application, when it instigates in a photographic subject image in the image 
used as criteria and distortion has arisen, it instigates also in the stuck synthetic image and there is a 
problem that distortion will be included. That is, the magnitude of the gate distortion in the synthetic 
image generated changes by on the basis of which image lamination is performed among two or more 
images which took a photograph in division and were obtained. It explains referring to drawing 1 about 
this problem. 

[0007] If these three images are stuck on the basis of the image IM 1 photoed and obtained from left 
slant to the photographic subject side PL when a certain photographic subject side PL is photoed from 
three ways D1-D3 and images IM1-IM3 are obtained, respectively as shown in drawing 1 (a) for 
example, the synthetic image IMA as shown in drawing 1 (b) will be obtained. Moreover, if it sticks 
similarly on the basis of the photoed image IM 2 from a transverse plane mostly, the synthetic image 
1MB as shown in drawing 1 (c) will be obtained. Here, as shown in drawing 1 (b) and drawing 1 (c), 
both the above-mentioned composition images IMA and 1MB differ in the magnitude of gate distortion 
greatly. 

[0008] Therefore, since a photograph is wanted to carry out the right pair of at least one in the photoed 
image of two or more sheets to the photographic subject side PL mostly, and to be taken, in case it is 
lamination among the photoed images of two or more sheets, when photoing the interface which 
chooses the image made into criteria, and the image used as the criteria of the lamination, an interface 
which demands cautions from a user is desired. 
[0009] 

[Problem(s) to be Solved by the Invention] This invention is made in view of an above-mentioned point, 
and aims at offering the record medium in which computer reading for realizing the image-processing 
approach, the image processing system, and this image-processing approach for amending distortion and 
obtaining a more proper image easily is possible. 
[0010] 

[Means for Solving the Problem] The above-mentioned purpose is the image-processing approach which 
amends distortion of the image photoed from two or more directions so that at least a part might overlap 
to a photographic subject. The first step which specifies the correspondence relation of the duplication 
part in two or more images obtained by photography, The second step which chooses the object which 
amends distortion from two or more images, It is attained by offering the image-processing approach 
characterized by having the third step which amends distortion of the image chosen at the second step 
according to the correspondence relation specified in the first step. Since the object which amends 
distortion is chosen according to such a means, in order to obtain a proper image, the optimal image can 
be made applicable to amendment. 

[001 1] You may make it choose the object of amendment automatically at the second step here 
according to the size of the field which a photographic subject occupies into an image. According to 
such a means, the paintings-and-calligraphic-works amount of information needed can amend distortion 
of the most abundant images automatically. 

[0012] Moreover, you may make it choose the object of amendment automatically at the second step 
according to the sense of the straight-line-like pattern detected in an image. According to such a means, 
the image photoed from the location which carried out the right pair to the photographic subject mostly 
can be automatically made applicable to amendment. 

[0013] Moreover, at the second step, if the object of amendment is automatically chosen according to 
the correspondence relation specified in the first step, high amendment of precision can be performed 
certainly. 

[0014] Moreover, at the second step, if the object of amendment is automatically chosen according to 
the sense of the photographic subject detected for every photography, the image photoed from the 
location which carried out the right pair mostly to the photographic subject can be automatically made 
applicable to amendment. 
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[0015] Moreover, the purpose of this invention is the image-processing approach which amends 
distortion of the image photoed from two or more directions so that at least a part might overlap to a 
photographic subject. The first step which specifies the correspondence relation of the duplication part 
in two or more images obtained by photography, respectively, The second step as which distortion 
chooses fewest images from two or more images, According to the correspondence relation specified in 
the first step, distortion of two or more images is amended, respectively, and it is attained by offering the 
image-processing approach characterized by having the image chosen in the second step, and the third 
step to compound. According to such a means, since a synthetic image is generated on the basis of an 
image with least distortion, a more proper synthetic image can be obtained. 

[0016] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A correspondence detection means to detect the correspondence relation of the duplication part 
in two or more images obtained by photography, It responds to the correspondence relation by which 
distortion was detected in a selection means to choose fewest images, and the correspondence detection 
means, out of two or more images. Distortion of two or more images is amended, respectively, and it is 
attained by offering the image processing system characterized by having the image chosen in the 
selection means, and an image composition means to compound. 

[0017] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A selection means to choose beforehand the image made into the criteria at the time of 
amending distortion among two or more images to be photoed from now on, A notice means to notify a 
user of the image photoed next turning into an image made into criteria according to the selection made 
by the selection means, A correspondence detection means to detect the correspondence relation of the 
duplication part between the image made into the criteria acquired by taking a photograph, and other 
images, It is attained by offering the image processing system characterized by having an amendment 
means to amend the distortion of an image made into criteria, according to the correspondence relation 
detected by the correspondence detection means. According to such a means, in the photography of an 
image made into the criteria at the time of amending distortion, attention can be called to a user. 
[0018] Moreover, the purpose of this invention is an image processing system which amends distortion 
of the image photoed from two or more directions so that at least a part might overlap to a photographic 
subject. A correspondence detection means to detect the correspondence relation of the duplication part 
in two or more images obtained by photography, It is attained by offering the image processing system 
characterized by having a selection means to choose the object which amends distortion from two or 
more images, and an amendment means to amend distortion of the image chosen by the selection means 
according to the correspondence relation detected by the correspondence detection means. 
[00 1 9] Moreover, two or more optical means which the purpose of this invention is an image processing 
system which amends distortion of the image photoed from two or more directions as at least a part 
overlaps to a photographic subject, and photo a photographic subject to coincidence, A selection means 
to choose the image made into the object of amendment among the images photoed by two or more 
optical means, A correspondence detection means to detect the correspondence relation of the 
duplication part between the image chosen by the selection means, and other images, It is attained by 
offering the image processing system characterized by having an amendment means to amend distortion 
of the selected image, according to the correspondence relation detected by the correspondence detection 
means. Since two or more photographic subject images once photoed by photography from two or more 
directions can be obtained according to such a means, the count of photography needed in order to 
amend distortion can be reduced. 

[0020] Here, a selection means shall choose the object of amendment automatically according to the 
sense of the straight-line-like pattern which shall choose the object of amendment automatically 
according to the size of the field which a photographic subject occupies into an image, or is detected in 
an image. Moreover, a selection means is good also as what chooses the object of amendment 
automatically according to the sense of the photographic subject which should choose the object of 
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amendment automatically according to the correspondence relation detected by the correspondence 
detection means, or was detected for every photography. 

[0021] Moreover, the purpose of this invention is a record medium which recorded the program for a 
computer to amend distortion of the image photoed from two or more directions so that at least a part 
might overlap to a photographic subject and in which computer reading is possible. This program makes 
the correspondence relation of the duplication part in two or more images obtained by photography 
specify to a computer. It is attained by offering the record medium which is characterized by making 
distortion of the image which was made to choose the object which amends distortion and was chosen 
according to the specified correspondence relation from two or more images amend and in which 
computer reading is possible. According to such a means, in order to obtain a proper image, the optimal 
image can be easily made applicable to amendment. 

[0022] Moreover, the purpose of this invention is a record medium which recorded the program for a 
computer to amend distortion of the image photoed from two or more directions so that at least a part 
might overlap to a photographic subject and in which computer reading is possible. This program makes 
the correspondence relation of the duplication part in two or more images obtained by photography 
specify to a computer, respectively. It is attained by offering the record medium which is characterized 
by making it compound with the image which was made to amend distortion of two or more images, 
respectively, and was chosen according to the correspondence relation as which distortion made fewest 
images choose as and it was specified from two or more images and in which computer reading is 
possible. According to such a means, a more proper synthetic image can be obtained easily. 
[0023] 

[Embodiment of the Invention] The gestalt of operation of this invention is explained in detail with 
reference to a drawing below. In addition, a same-among drawing sign shows the same or a considerable 
part. 

[Gestalt 1 of operation] drawing 2 is drawing for explaining the image-processing approach and image 
processing system concerning the gestalt 1 of operation of this invention. As the gestalt of this operation 
is shown to drawing 2 by here, the images 3 and 4 of two sheets are photoed so that a part of 
photographic subject side [ at least ] PL may overlap with an image processing system 1, and it is 
explained by the example of amending the gate distortion by either 3, for example, an image, among the 
images 3 and 4 of these two sheets, and finally obtaining the distortion amendment image 5. 
[0024] Drawing 3 is drawing showing the configuration of the image processing system 1 concerning 
the gestalt 1 of operation of this invention. As shown in drawing 3 , an image processing system 1 is 
equipped with the image pick-up section 11, the signal-processing section 12, the memory control 
section 13, the main control section 14, a frame memory 15, an interface 16, a display 17, the enternal 
memory section 18, the photography mode setting section 19, the criteria image setting section 20, the 
correspondence detecting element 21, and the image-distortion amendment section 22. And the image 
pick-up section 1 1 contains a lens 111, diaphragm 1 12, a shutter 1 13, an optoelectric transducer 1 14, 
and the pretreatment section 115. 

[0025] Here, the signal-processing section 12 is connected to the pretreatment section 115, the memory 
control section 13, the main control section 14, and an interface 16. Moreover, the memory control 
section 13 is further connected to a frame memory 15 and the criteria image setting section 20. The main 
control section 14 is further connected to the memory control section 13, the photography mode setting 
section 19, and the criteria image setting section 20. 

[0026] Moreover, a frame memory 15 is connected to the memory control section 13, the 
correspondence detecting element 21, and the image-distortion amendment section 22. Moreover, an 
interface 16 is further connected to a display 17 and the enternal memory section 18. And the criteria 
image setting section 20 is further connected to the image-distortion amendment section 22. Moreover, 
the correspondence detecting element 2 1 is further connected to the image-distortion amendment section 
22. 

[0027] On the other hand, in the image pick-up section 1 1, it extracts as a lens 111, and 1 12, a shutter 
113, and an optoelectric transducer 114 are arranged on an optical axis at this order, and an optoelectric 
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transducer 1 14 is connected to the pretreatment section 115. 

[0028] In the above, photography mode is changed by the photography mode setting section 19, and the 
image which instigates in the criteria image setting section 20, and amends distortion is set up. 
Moreover, the correspondence detecting element 21 extracts the focus and corresponding points between 
both images in two images with which at least the part overlapped mutually. And the image-distortion 
amendment section 22 amends the gate distortion in the photoed image according to the signal supplied 
from the correspondence detecting element 2 1 . In addition, a setup of the criteria image in the above, 
actuation of the correspondence detecting element 2 1 , and amendment of gate distortion are explained in 
detail later. 

[0029] Moreover, CCD is used for the optoelectric transducer 1 14 of the image pick-up section 1 1 . 
Moreover, the pretreatment section 1 1 5 is equipped with the analog signal processing section and the 
analog-digital converter (A/D converter) which consist of pre amplifier, an automatic gain control circuit 
(Auto Gain Control- AGC), etc., and after pretreatment of magnification, a clamp, etc. is performed to 
the analog video signal outputted from the optoelectric transducer 114, the above-mentioned analog 
video signal is changed into a digital video signal. 

[0030] Moreover, the signal-processing section 12 is constituted by the digital signal processor (DSP 
processor) etc., and performs various image processings, such as color separation, white balance 
adjustment, and gamma amendment, to the digital video signal acquired in the image pick-up section 1 1 . 
Moreover, the memory control section 13 stores in a frame memory 15 the picture signal processed by 
doing in this way, or reads the picture signal conversely stored in the frame memory 15. Moreover, the 
main control section 14 is constituted by the microcomputer etc. Moreover, a frame memory 15 stores 
the image of at least two sheets, and, generally semiconductor memory, such as VRAM, SRAM, and 
DRAM, is used. 

[003 1 ] Here, the picture signal read from the frame memory 1 5 is saved in the entemal memory section 
18 through an interface 16, after signal processing, such as picture compression, is performed in the 
signal-processing section 12. Reading and this entemal memory section 18 write various signals, such as 
a picture signal supplied through an interface 16, and is constituted by IC memory card, the magneto- 
optic disk, etc. As the enternal memory section 18, if a modem card and an ISDN card are used, a 
picture signal can also be transmitted to the record medium of a direct remote place via a network here. 
[0032] Moreover, a picture signal is transmitted to the signal-processing section 12 through an interface 
1 6, and read-out of the picture signal conversely recorded on the entemal memory section 1 8 is 
performed by giving image expanding in the signal-processing section 12. On the other hand, the display 
of the picture signal read from the entemal memory section 1 8 and a frame memory 1 5 is performed by 
transmitting to a display 17 through an interface 16, after performing signal processing, such as digital 
to analog (D/A conversion) and magnification, to a picture signal in the signal-processing section 12. A 
display 17 consists of liquid crystal displays which displayed the image according to the picture signal 
supplied through the interface 16, for example, were installed in the case of an image processing system 
1 here. 

[0033] Drawing 4 is the perspective view showing the image processing system shown in drawing 3 . As 
shown in drawing 4 , the image processing system 1 concerning the gestalt of this operation contains an 
electric power switch 101, a shutter 102, a finder 103, the photography mode setting key 104 for setting 
photography mode as the photography mode setting section 19, the above scrolling key 201 for scrolling 
upward the image projected on the display 17, the down scrolling key 202 for scrolling downward the 
image projected on the display 17, and the decision key 203. 

[0034] It explains referring to the flow chart shown in drawing 4 and drawing 5 in actuation of the 
image processing system which has the above configurations below. First, an electric power switch 101 
is changed, an image processing system 1 is started, and photography mode is chosen. Here, the above- 
mentioned photography mode consists of the normal mode which takes the usual snapshot, and gate 
amendment mode which generates the image which amended the gate distortion of the photoed image. 
And selection in this photography mode is made when a user operates the photography mode setting key 
104. In addition, as the photography mode setting section 19, the photography mode setting key 104 is 
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formed in the body of an image processing system 1. However, the photography mode setting section 19 
may consist of hardware or software etc. which are formed separately from a body. 
[0035] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 5 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0036] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And a user photos at least two photographic subject sides PL by the image pick-up section 11, 
and a photographic subject image is incorporated by the image processing system 1. In addition, each 
image needs to be photoed so that some photographic subject images may overlap mutually at this time. 
[0037] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. In addition, the above-mentioned 
directions are made by pressing the photography mode setting key 1 04 once again, and changing to the 
normal mode, and also they may form the switch for directing photography termination separately. 
[0038] Moreover, as shown during photography at drawing 6 , the screen overlay of the present 
photography number of sheets, such as "the 1st sheet 11 , and the photography termination approaches, 
such as "ending, if a photography mode setting key is pressed", may be carried out to a display 17. 
[0039] And although the actuation which instigates below to step S4 and amends distortion is started 
after photography is completed as mentioned above, it chooses the gate distortion of which image is first 
amended in step S4, and this image is set up. In addition, the image chosen at this time is also called a 
"criteria image" to below. And a setup of this criteria image is performed in the criteria image setting 
section 20. The configuration and actuation of the criteria image setting section 20 are explained in 
detail below. 

[0040] Drawing 7 is drawing showing the layout of the criteria image setting section 20 shown in 
drawing 3 . As shown in drawing 7 , the criteria image setting section 20 contains the above scrolling 
key 201, and the down scrolling key 202 and the decision key 203. And as shown in drawing 7 , when 
directions of photography termination are made by the user in step S3, in a display 17, an overlay 
indication of the alphabetic character "criteria image setup" is given, and it points so that a criteria image 
may be chosen as a user. 

[0041] Then, the above scrolling key 201 and the down scrolling key 202 are operated by the user, and it 
is displayed, the image photoed in the above-mentioned step S2 changing one by one. In addition, the 
image photoed after [ of the image by which it is indicated by current ] one is displayed by pressing the 
above scrolling key 201 by displaying the image photoed before [ of the image by which it is indicated 
by current ] one, and pressing the down scrolling key 202. And if the decision key 203 is pressed in the 
condition that the image chosen as a criteria image is displayed on the display 17, the image currently 
displayed at the time will be determined as a criteria image. 

[0042] And the image with the smallest possible angle that crossed to the field where the photographic 
subject side PL is the largest, and has been reflected to it in two or more photoed images as the above- 
mentioned criteria image, and is made to the photographic subject side of the image pick-up side at the 
time of photography is chosen by the user. 

[0043] Here, choosing automatically the image optimal as the object which calculates a photographic 
subject field and the above-mentioned tilt angle inside equipment, instigates according to the result of 
this count, and amends distortion, i.e., a criteria image, is also considered. Drawing 8 is drawing 
showing the configuration of the image processing system 2 which realizes such actuation. As shown in 
drawing 8 , an image processing system 2 is equipped with the photographic subject field decision 
section 23 instead of the criteria image setting section 20 contained in the image processing system 1 
shown in drawing 3 . 

[0044] And as the above-mentioned photographic subject field decision section 23 performs processing 
which detects the field which a photographic subject occupies in the photoed image, for example, it is 
indicated by reference "processing of an image and recognition" (Takeshi Akoin and Tomoharu Nagao 
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collaboration, Shokodo) (a) How to cluster on an image like a field grown method or a field split plot 
experiment, (b) The field division approaches using the edge in images, such as the approach of 
clustering on feature spaces, such as field division by the histogram, and (c) profile line tracking, such as 
an approach and the (d) texture analysis, are applied. However, when a photographic subject side is a 
rectangle, the field of a photographic subject is determined as a meaning by carrying out the external 
input of the top-most-vertices coordinate of the four corners of the photographic subject on an image. 
[0045] Thus, the data in which the field of the photographic subject acquired in the photographic subject 
field decision section 23 is shown are supplied to the image-distortion amendment section 22, and an 
image with this largest field is chosen in the image-distortion amendment section 22. Furthermore, in the 
image-distortion amendment section 22, since the include angle made to the photographic subject side 
PL of an image pick-up side is calculated, when there are two or more images with the above-mentioned 
largest field (i.e., when there are two or more images with which for example, the whole photographic 
subject side is reflected), finally an image with the above-mentioned smallest include angle is chosen as 
a criteria image, so that it may explain in full detail below. 

[0046] In addition, the above functions which choose a criteria image automatically are applicable 
similarly in the gestalt of which [ of the following ] operation. 

[0047] Next, it progresses to step S5, the focus is detected in the criteria image determined as mentioned 
above, and the corresponding points which show the same part as the above-mentioned focus in the 
image (it is also called a "reference image" to below.) which overlaps in these some criteria images [ at 
least ] are detected. Detection of such the focus and corresponding points is performed by the 
correspondence detecting element 2 1 shown in drawing 3 . Then, the configuration and actuation of this 
correspondence detecting element 21 are explained in detail below. 

[0048] The correspondence detecting element 21 detects the photoed same part in the image with the 
field which overlapped mutually as mentioned above of two sheets. And the approach which used the 
correlation operation here is explained. 

[0049] Drawing 9 is drawing showing the configuration of the correspondence detecting element 21 
shown in drawing 3 . As shown in drawing 9 , the correspondence detecting element 2 1 is equipped with 
the focus setting section 211 connected to the frame memory 15, and the correlation operation part 212 
connected to the focus setting section 211 and a frame memory 15. In addition, the criteria image and 
the reference image are stored in the frame memory 1 5 shown in drawing 3 . 

[0050] After the focus setting section 211 determines the location of the focus in a criteria image, it 
extracts the shade pattern of x (2N+1) (2P+1) individual centering on the focus, and creates the data of 
the field called a correlation aperture here. In addition, the location of the above-mentioned focus is 
determined by extracting the part where the concentration pattern of an image is characteristic like an 
angle (comer). 

[005 1 ] Moreover, the correlation operation part 212 detects the part which is mostly in agreement with 
the shade pattern of the correlation aperture created based on the criteria image by performing a 
correlation operation in a reference image, and determines this as corresponding points. An example 
which detects corresponding points by block matching by the correlation operation here is explained 
referring to drawing 10 . 

[0052] As shown in drawing 10 , in block matching of the correlation aperture 215,216 which consists 
of a shade pattern of x (2N+1) (2P+1) individual, the cross-correlation value Si of the i-th focus 213 
which has a coordinate in the criteria image 7 (xiO, yiO), and the corresponding points 217 which have a 
coordinate in the reference image 9 (xiO+dxi, yiO+dyi) is calculated by the degree type. 
[0053] 
[Equation 1] 
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In addition, in the above-mentioned formula (1), Is (x y) shows concentration [ in / for the concentration 
in the coordinate point (x y) of the criteria image 7 / in an example and Ir (x y) / the coordinate point (x 
y) of the reference image 9 ]. Moreover, Is (x y) shows the average concentration in the pattern of x 
(2N+1) (2P+1) individual centering on the coordinate point in the /correlation aperture 215 in the criteria 
image 7 (x y). Ir (x y) shows the average concentration in the pattern of x (2N+1) (2P+1) individual 
centering on the coordinate point in the /correlation aperture 216 in the reference image 9 (x y). 
Moreover, K shows a constant. 

[0054] And the corresponding points 217 in the reference image 9 are called for by searching for the 
point which is beyond the threshold as which the maximum of the cross-correlation value Si was 
beforehand determined by the above-mentioned formula (1) to each focus 213. In addition, if the 
maximum of the cross-correlation value Si becomes below a threshold, corresponding points shall not 
exist. 

[0055] Thus, after detection of the focus and corresponding points is completed, while calculating the 
parameter which amends the gate distortion of the criteria image 7 in step S6 shown in drawing 5 , the 
image which amended the gate distortion of an image based on this parameter in step S7 is created, and 
actuation is ended. In addition, the image which amended a "distortion amendment parameter" and gate 
distortion for the above-mentioned parameter below, respectively is also called "distortion amendment 
image." 

[0056] And count of the above-mentioned distortion amendment parameter and generation of a 
distortion amendment image are performed by the image-distortion amendment section 22. The 
configuration and actuation of this image-distortion amendment section 22 are explained in detail below. 

[0057] The image-distortion amendment section 22 amends gate distortion by changing into the image 
which photoed the photographic subject side from the transverse plane using the relation of the focus 
and corresponding points which the correspondence detecting element 2 1 detected. And the 
configuration of this image-distortion amendment section 22 is shown in drawing 1 1 . As shown in 
drawing 1 1 , the image-distortion amendment section 22 contains the three-dimension operation part 
221, the parameter calculation section 222, and the coordinate transformation section 223. Here, the 
three-dimension operation part 221 is connected to the correspondence detecting element 21 and the 
criteria image setting section 20, and the parameter calculation section 222 is connected to the three- 
dimension operation part 221. Moreover, the coordinate transformation section 223 is connected to the 
parameter calculation section 222, a frame memory 15, and the criteria image setting section 20. 
[0058] Actuation of the image-distortion amendment section 22 is explained below. In addition, as 
shown below at drawing 12 , while the criteria image 7 and the reference image 9 are photoed to the 
photographic subject side PL As the optical system of the image pick-up section 1 1 is shown in drawing 
13 , it is related with a x axis. Facing the right of the image side 224 Forward, The sense which goes 
facing down of the image side 224 to the image side 224 about the z-axis of forward and the direction of 
an optical axis about the y-axis from the zero O which is the optical axis of the image pick-up section 1 1 
Forward, The focal distance of this optical system explains as an example the case where it considers as 
the central projection model (perspectiveprojection model) set to f. 

[0059] The three-dimension operation part 221 shown in drawing 1 1 computes the following three- 
dimension parameters with the relation between the above-mentioned focus 213 and corresponding 
points 217. That is, normal vector n which shows the sense of the translational-motion vector t which 
shows change of the location of the image pick-up section 1 1 at the time of the reference image 
photography based on the time of the rotation matrix R which shows change of the sense of the image 
pick-up section 1 1 at the time of the reference image photography based on the time of the criteria image 
photography shown in drawing 12 , and criteria image photography, and the photographic subject side 
PL is computed. And the method of asking for these three three-dimension parameters {R, t, n} is 
mainly made into two of the followings. 

[0060] that is, as a primary method, by the group of the eight or more focus and corresponding points, 
after calculating the location of the camera at the time of each image photography, a posture, and the 
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three-dimension coordinate of each corresponding points, it assumes that a photographic subject is a flat 
surface, and there is a method of applying this profit **** three-dimension coordinate to one flat 
surface. 

[0061] Moreover, as the second approach, from the group of the four or more focus and corresponding 
points, a projective-transformation matrix (homography matrix) is calculated and there is the approach 
of computing the location of the camera at the time of each image photography and the sense of a 
posture and a photographic subject side according to the acquired projective-transformation matrix. 
[0062] Here, although the above-mentioned primary method is a general-purpose movement 
stereoscopic vision technique and a meaning is asked for the above-mentioned parameter {R, t, n} by 
the linearity operation, the detail is recorded on the common reference (for example, "three-dimension 
vision" **** and crossing Saburo collaboration, KYORITSU SHUPPAN) about three-dimension 
measurement or a computer vision. On the other hand, the second approach of the above computes the 
sense of a camera, and the sense of a photographic subject side, after asking for the coordinate 
transformation equation (projective-transformation matrix) materialized under the constraint that a 
photographic subject is a flat surface. And although the three-dimension operation part 221 can, needless 
to say, take any [ the above-mentioned first and / second ] approach, it explains the actuation based on 
the second approach here. 

[0063] The calculation procedure of a projective-transformation matrix is explained in detail first. Here, 
it points out obtaining the image 10 changed into the image obtained when the photographic subject 
image reflected to the criteria image 7 is photoed from the same direction as the reference image 9, as 
indicated in drawing 14 as the projective transformation from a criteria image to a reference image. And 
when this projective transformation is expressed with a formula and the point (xs, ys) in a criteria image 
and the point (xr, yr) in a reference image have a correspondence relation, it becomes like a degree type. 

[0064] 
[Equation 2] 

And they are eight unknowns bl-b8 in the above-mentioned formula (2) [0065] 
[Equation 3] 
~b x b, b, 



b A b s b 6 
LA I* I 



(3) 



It collects as a matrix B and this is called a projective-transformation matrix. In order to search for this 
projective-transformation matrix B, 4 or more sets of groups of the focus whose coordinates in a criteria 
image are (xsi, ysi), and the corresponding points (i= 1, --, N;N>=4) whose coordinates in a reference 
image are (xri, yri) are used. Here, although what is necessary is to substitute a coordinate (xsi, ysi) and 
a coordinate (xri, yri) to the above-mentioned formula (2), and just to calculate the solution of bl-b8, 
since a formula (2) is not materialized according to errors, such as a noise superimposed on an image in 
fact, it will solve using the least square operation shown below. 
[0066] 
[Equation 4] 



E 



&7*„ + b t y ti + 1 



kx M + b t y„ + 1 



mm . (4) 
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And the above-mentioned formula (4) deforms as follows. 

[0067] 

[Equation 5] 

E tte ** + to* + *i - fa** + by* + i>w } 1 



-» min. 



(5) 



Use of the constraint that the value of the derivative obtained by carrying out strange differential of the 
left part of the above-mentioned equation (5) by bl-b8, respectively is set to 0 calculates bl-b8 by 
solving a simultaneous equation. That is, the projective-transformation matrix B can be searched for by 
the easy linearity operation using the group matched the account of a top. 

[0068] Then, the procedure of asking for a three-dimension parameter {R, t, n} from the projective- 
transformation matrix B is explained. It is normal vector n of a photographic subject side [0069] 
[Equation 6] 



n = 



b 

c 



(a 2 +A 2 + c 2 =l. c>6) 



(6) 



It is the equation of the photographic subject [ carry out and ] side on the basis of the time of criteria 
image photography [0070] 
[Equation 7] 

(/i.r)+d = 0 (7) 



Here, |d| shows the distance from the zero to the photographic subject side PL, and is r=[x y z] T. It sets. 
Moreover, a formula (2) is rewritten like a degree type using a focal distance f. 
[0071] 
Equation 8] 

*' H u x a +H 7 >y 9 + H 3J f (w 



Furthermore, a formula (8) is [0072]. 
[Equation 9] 
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(9) 



It deforms. However, [0073] 
[Equation 10] 

3 H t3 x, + H„y, + H„f 



(10) 
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It comes out. At this time, the matrix H of a formula (9) and the relation with a parameter {R, t, n, d} 

become like a degree type. 

[0074] 

[Equation 1 1 ] 
ff r =s'R r (d/ + tn r ) 



00 



However, s' is a constant and each element of Matrix H has one times the degree of freedom of a scale. 
Moreover, a degree type can perform conversion in procession H of a formula (9) from the projective- 
transformation matrix B of a formula (3). 
[0075] 
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(12) 
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The procedure of asking below for a parameter {R, t, n, d} stranger than Matrix H is shown. However, 
since it is unfixed, Variable d and the scale of the magnitude of the translational-motion vector t 
showing distance with a photographic subject side are [0076]. 
[Equation 13] 

lltlNl (13) 

It assumes. Here, although the detail is describing in reference "mathematical principle of image 
comprehension-three-dimension recognition -" (Ken-ichi Kaneya work, Morikita Shuppan), the 
computation which derives the solution {R, t, n, d} to is as follows when the result is summarized, 
(i) It is made to be set to det[H] =1, applying the suitable constant for each element of the matrix H of a 
formula (9). (ii) Characteristic value of a symmetric matrix HHT is set to sigmal2, sigma22, and 
sigma32, the corresponding characteristic vectors ul, u2, and u3 are mutually intersected 
perpendicularly, and it takes to the unit vector which makes a right-hand system in this order. However, 
sigmal>=sigma2>=sigma3> It is referred to as 0. (iii) A movement parameter will be [0077] if it 
becomes sigma 1 =sigma2=sigma3 . 
[Equation 14] 

t=0, R = H (14) 

It comes out, and it is and the parameter {n, d} of a photographic subject side is unfixed. Otherwise, 2 
sets of solutions are acquired as follows, (iv) The parameter {n, d} of a photographic subject side 
becomes settled as follows. 
[0078] 

[Equation 15] 



n = 



V^ 1 



(15) 



[0079] 

[Equation 16] 
d = 



(16) 



However, it is epsilon= **1, and epsilon is chosen so that it may be set to c> 0. Moreover, (v) unit 

translational-motion vector t becomes settled as follows. 

[0080] 
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[Equation 17] 

t J r (b«r 3 V5? 



&2 



<t\u x 



-cr t JaZ -(tIu 3 ) 



(17) 



Moreover, the rotation matrix R becomes settled as follows. 
[0081] 

[Equation 18] 



cr_[ (n,t)+d J 



(18) 



Therefore, although two kinds of solutions {R, t, n, d} to are acquired from Matrix B, when the most, a 
true solution can be distinguished from the drawn value. In addition, although the focal distance f of the 
image pick-up section 1 1 is used by intermediate count, the value of a focal distance f can be easily 
obtained by the approach of memorizing the optical-system parameter of the image pick-up section 1 1 to 
an internal memory (not shown). Moreover, if the focal distance of both images is known even when the 
focal distances of a criteria image and a reference image differ [ the focal distance of the optical system 
of the image pick-up section 1 1 ] with adjustable, the above-mentioned three-dimension parameter 
calculation procedure can be applied as it is, and a focal distance can be detected by the approach of 
installing an encoder in optical system. 

[0082] Next, the parameter calculation section 222 shown in drawing 1 1 calculates the parameter which 
amends gate distortion based on relation with the sense of the photographic subject side computed by 
the image pick-up section 1 1 and the three-dimension operation part 221 when photoing a photographic 
subject side. As the gestalt of this operation is shown to drawing 15 by here, the gate distortion of an 
image is amended by performing projective transformation of projecting the image side 30 containing 
the photographic subject image which has gate distortion, by using as the plane of projection 31-ed a flat 
surface parallel to the photographic subject side which the three-dimension operation part 221 
computed. The point PI on the image side 30 is projected on the point P2 on the plane of projection 31- 
ed here. The count approach of the parameter which amends the above-mentioned gate distortion below 
is explained. 

[0083] First, as shown in drawing 16 , it asks for rotation matrix R' which shows the coordinate 
transformation which makes the z-axis in the device-coordinate system 33 in agreement with the unit 
normal vector of the plane of projection 31-ed. In this case, the following relational expression is 
materialized. 
[0084] 





0" 
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R' 


0 




b 
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L - 




c 

m — 



(19) 



And although much rotation matrix R' which fills the above-mentioned formula (9) exists, rotation 

matrix R' is defined like a degree type here. 

[0085] 

[Equation 20 





R» 










RU 







(20) 



However, R'x and R'y are shown as follows here, respectively. 
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[0086] 

[Equation 21] 
1 0 



0 cosa 
0 since 



0 

-sina 
cosa 



K = 



' cos/3 0 sin/? 

0 I 0 
-sin/3 0 cos/3 



(21) 



As shown in drawing 17 , this rotates the device-coordinate system (xyz system of coordinates) 33 in 
following sequence, and is equivalent to changing into x'y'z 1 system of coordinates, (i) beta rotation of 
the device-coordinate system 33 is done only around the y-axis. And let the system of coordinates 
obtained by this rotation be lylzxl system of coordinates, (ii) Only alpha rotates a device-coordinate 
system around xl shaft. 

[0087] Here, if a formula (19) and a formula (20) are used, an angle of rotation will be drawn like a 

degree type. 

[0088] 

[Equation 22] 
a = sin -1 (- b) 



p — sin 



Va 2 + c 2 j 



(22) 



(23) 



And matrix R f can be set to a meaning by substituting for a formula (20) and a formula (21) the angle of 
rotation searched for in this way. 

[0089] Next, coordinate transformation of the coordinate on the image side 30 is carried out on the plane 
of projection 3 1-ed. That is, let the point P2 which intersects the plane of projection 3 1 -ed when the 
three-dimension vector p corresponding to the point PI of the image side 30 is extended be a coordinate 
after coordinate transformation in drawing 15 . And the three-dimension vector p corresponding to the 
point PI on the basis of the device-coordinate system 33 is shown by the degree type. 
[0090] 

[Equation 23] 



ax, + by, 



(k>0) 



(24) 



Here, since k is a scaling factor showing the distance from the optical axis o of the image pick-up 
section 1 1 to plane of projection-ed, k expresses the magnitude of the distortion amendment image 
created, moreover — a three dimension — a vector — p — criteria — an image — photography — the time 
a device coordinate — a system — 33 — criteria — carrying out — a point — P — one ~ having expressed 
a vector — it is — although — this — rotation — a matrix — R — 1 — using — a degree — a type — like — 
coordinate transformation — carrying out — things — an image pick-up — the section — 1 1 — a 
photographic subject — a field — right — a pair — carrying out — having made — the time — a three 
dimension — a vector — p — 1 — changing — having . 
[0091] 



[Equation 24] 



p'= 
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Z 



^3 



^21 
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(25) 



Therefore, the image which amended gate distortion is obtained by making the x-coordinate and y- 
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coordinate of a formula (25) into the coordinate acquired after coordinate transformation. And the 
parameter which amends the gate distortion of a criteria image using a formula (25) is computable with 
the above procedures. 

[0092] Next, the coordinate transformation section 223 coordinate-transformation-creates a distortion 
amendment image for a criteria image based on the gate distortion amendment parameter computed by 
the parameter calculation section 222. The coordinate (xs, ys) before the conversion corresponding to 
the coordinate after coordinate transformation (X, Y) is calculated based on a formula (25), and, 
specifically, a interpolation operation determines the pixel value in a coordinate (X, Y) based on a pixel 
value [ / near the calculated coordinate (xs, ys) ]. In addition, what is necessary is just to perform this 
interpolation operation using the existing approaches, such as a congruence linear interpolation method 
and B-spline interpolation method. 

[0093] As mentioned above, when according to the image processing system concerning the gestalt 1 of 
operation of this invention a photographic subject side is photoed two or more sheets so that at least 
parts may overlap mutually, the gate distortion in the image of the photoed arbitration can be amended, 
and the overview of a more proper photographic subject can be obtained. 

[0094] That is, it is sufficient, if there are an image which photoed the whole photographic subject, and 
an image which photoed some photographic subjects as it is necessary to photo all the range of a 
photographic subject in no images in the above-mentioned procedure for example, and is shown in 
drawing 14 . And since the image which photoed the whole photographic subject by the criteria image 
setting section 20 can be chosen as a criteria image in such a case, the overview of the photographic 
subject with which gate distortion was amended is generable. 

[0095] Moreover, if the number of sheets to photo is limited to two sheets since there should just be a 
photographic subject image of two sheets at worst in order to generate a distortion amendment image by 
the above-mentioned approach, count cost which the input of an image and a setup of photography and a 
criteria image are not only simplified for a user, but detection of memory space, the focus, and 
corresponding points required in order to memorize an image, and count of a projective-transformation 
matrix take can be made small. 

[Gestalt 2 of operation] drawing 18 is drawing for explaining the image-processing approach and image 
processing system concerning the gestalt 2 of operation of this invention. As shown in drawing 18 (a), 
actuation of the image processing system 6 concerning the gestalt of this operation when the static image 
of the same photographic subject side PL is photoed from two or more directions dl-dk so that it may 
overlap in some images, respectively is explained. 

[0096] Here, as shown in drawing 18 (a), suppose that there is a duplication field, respectively by taking 
a photograph from Direction Dn by Hazama of Image imj and Image im (j+l) (1 <=j<=k -1) which were 
obtained by taking a photograph in the direction which Image imn (n= 1,2—, j, — , k) is obtained, for 
example, adjoins each other like an image iml and an image im2. 

[0097] And in the image processing system concerning the gestalt 2 of this operation, as shown in 
drawing 18 (b), other images are stuck and the synthetic image IMC is obtained so that it may have 
consistency in the gap or one image imj chosen as a criteria image. 

[0098] Drawing 19 is drawing showing the configuration of the image processing system 6 concerning 
the gestalt 2 of this operation. As shown in drawing 19 , although an image processing system 6 has the 
same configuration as the image processing system 1 concerning the gestalt 1 of the above-mentioned 
implementation shown in drawing 3 , it is different in that it has the image composition section 24 
instead of the image-distortion amendment section 22. Here, although the image composition section 24 
sticks an image with the field which overlaps mutually by carrying out coordinate transformation based 
on the relation of the focus and corresponding points which were obtained by the correspondence 
detecting element 21, it is later explained in detail about this actuation. 

[0099] Drawing 20 is a flow chart which shows actuation of the image processing system 6 concerning 
the gestalt 2 of this operation shown in drawing 19 . First, an electric power switch 101 is changed, an 
image processing system 1 is started, and photography mode is chosen. Here, the above-mentioned 
photography mode consists of the normal mode which takes the usual snapshot, and gate amendment 
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mode which generates the image which amended the gate distortion of the photoed image. And selection 
in this photography mode is made when a user operates the photography mode setting key 104. In 
addition, as the photography mode setting section 19, the photography mode setting key 104 is formed 
in the body of an image processing system 1. However, the photography mode setting section 19 may 
consist of hardware or software etc. which are formed separately from a body. 
[0100] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 5 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0101] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And a user photos at least two photographic subject sides PL by the image pick-up section 11, 
and a photographic subject image is incorporated by the image processing system 6. In addition, each 
image needs to be photoed so that some photographic subject images may overlap mutually at this time. 
[0102] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. In addition, the above-mentioned 
directions are made by pressing the photography mode setting key 104 once again, and changing to the 
normal mode, and also they may form the switch for directing photography termination separately. 
[0103] And after photography of a photographic subject is completed as mentioned above, the actuation 
which generates the synthetic image which stuck the image of two or more sheets below on step S4 is 
started. In step S4, it chooses first on the basis of which image a synthetic image is generated. In 
addition, the image chosen at this time is the above-mentioned criteria image, and the case where the 
image imj shown in drawing 18 as an example is chosen as below as a criteria image is explained. 
Moreover, although a setup of this criteria image is made in the criteria image setting section 20, the 
configuration and actuation of the criteria image setting section 20 of it are the same as that of the case 
in the gestalt 1 of the above-mentioned implementation. 

[0104] Next, in step S5, the image pair photoed from the adjoining direction shown in drawing 18 (a), 
i.e., the corresponding points which show the same part as the focus and this focus between images n 
(n+1) (1 <=n<=k -1), is detected. Detection of this focus and corresponding points is performed by the 
correspondence detecting element 21. Here, the configuration and actuation of the correspondence 
detecting element 21 are the same as that of the case in the gestalt 1 of the above-mentioned 
implementation. 

[0105] And termination of detection of the focus and corresponding points sticks an image, after 
carrying out coordinate transformation based on the relation of both the acquired points so that it may 
have consistency in a criteria image. Here, when using projective transformation as coordinate 
transformation, while calculating a projective-transformation matrix in step S6, a synthetic image is 
generated by the above-mentioned lamination in step S7. Such calculation of a projective-transformation 
matrix and generation of a synthetic image are performed by the image composition section 24. The 
configuration and actuation of this image composition section 24 are explained in detail below. 
[0106] Drawing 2 1 is drawing showing the configuration of the image composition section 24 shown in 
drawing 19 . As shown in drawing 2 1 , the image composition section 24 contains the projective- 
transformation calculation section 231 and the coordinate transformation section 232. Here, the 
projective-transformation calculation section 231 is connected to the criteria image setting section 20 
and the correspondence detecting element 21, and the coordinate transformation section 232 is 
connected to the criteria image setting section 20, a frame memory 15, and the projective-transformation 
calculation section 23 1 . 

[0107] And the projective-transformation calculation section 231 computes the projective- 
transformation matrix B shown by the formula (3) using the group of the four or more focus and 
corresponding points. The calculation procedure is the same as that of the case in the gestalt 1 of the 
above-mentioned implementation, and should just perform least square count of a formula (5). It is 
necessary to calculate the projective-transformation matrix between the image pairs of an individual (k- 
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1) by setting in the gestalt of this operation, and to search for the projective-transformation matrix from 
each image to a criteria image further here. 

[0108] As shown in drawing 18 , more specifically, they are Bn and a projective-transformation matrix 
from Image imn to Image imj about the projective-transformation matrix from Image imn (n= 1 to k-1) 
to Image im (n+1) [0109] 
[Equation 25] 

If it sets, this projective-transformation matrix is calculable based on a degree type. 
[0110] 

[Equation 26] 

0^ (n<J) 

(26) 




(n> j) 



Here, the coordinate transformation section 232 is stuck [ Image / n ] in the image [ criteria ] imj based 
on the projective-transformation matrix shown by the formula (26) computed by the projective- 
transformation calculation section 231. Specifically, the coordinate (x y) before the coordinate 
transformation in Image n corresponding to the coordinate after the coordinate transformation in a 
criteria image (X, Y) is first calculated based on a degree type. 
[0111] 

^Equation 271 
x 



y 
i 



X 
Y 



(27) 



However, s is a constant for setting the third component of the column vector of the left part in a 
formula (27) to 1. Next, a interpolation operation determines the pixel value of a coordinate (X, Y) 
based on a pixel value [ / near the coordinate before coordinate transformation (x y) ]. And what is 
necessary is just to perform this interpolation operation using the existing approaches, such as a 
congruence linear interpolation method and B-spline interpolation method. 

[0112] As mentioned above, according to the image processing system 6 concerning the gestalt of this 
operation, the photographic subject side PL is photoed two or more sheets, and the synthetic image 
crossed to the large field of the photographic subject side PL is generated by sticking other images to the 
selected criteria image so that parts may overlap mutually. In this case, by the criteria image setting 
section 20, since the image photoed from the direction which carries out a right pair mostly to a 
photographic subject, i.e., the small image of gate distortion, can be chosen as a criteria image, it can 
instigate as a result and the small overall photographic subject image of distortion can be obtained as the 
above-mentioned synthetic image. 

After setting to the image processing system concerning the gestalten 1 and 2 of the [gestalt 3 of 
operation] above-mentioned implementation and inputting or photoing two or more photographic 
subject images previously, the criteria image which is made into the object of distortion amendment or is 
made into the criteria of a synthetic image is chosen. On the other hand, before the image processing 
system concerning the gestalt 3 of this operation photos a photographic subject, the above-mentioned 
criteria image is set up beforehand. 

[01 13] Drawing 22 is drawing showing the configuration of the image processing system 8 concerning 
the gestalt 3 of operation of this invention. As shown in drawing 22 , although the image processing 
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system 8 concerning the gestalt of this operation has the same configuration as the image processing 
system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at the point further 
equipped with the notice section 26. Here, the notice section 26 is connected to the main control section 
14 and the criteria image setting section 20. 

[0114] In addition, the shutter 113 and interface 1 6 which were included in this image processing system 
8 are connected to the main control section 14, and a finder 241 is connected to an interface 16. 
[01 15] Next, actuation of the image processing system 8 concerning the gestalt 3 of operation of this 
invention is explained, referring to the flow chart of drawing 23 . First, if it is judged whether it 
instigates in step SI and amendment mode is chosen, and gate amendment mode is not chosen but the 
normal mode is chosen, it will progress to step S10 and the snapshot of a desired photographic subject 
will be taken by the user. 

[01 16] On the other hand, if it instigates in step SI and amendment mode is chosen, it will progress to 
step S2. And in step S2, the image made applicable [ of gate distortion ] to amendment, i.e., a criteria 
image, is set as the criteria image setting section 20. The criteria image setting section 20 contains a 
cursor key 204 and the decision key 203 here, as shown in drawing 24 . And while an overlay indication 
of the alphabetic character "criteria image setup" is given at a display 17, it points so that the number of 
sheets photoed to a user after this and the criteria image assignment value which specifies whether the 
image photographed to the how many sheets in this photography is used as a criteria image may be set 
up. Here, a user can set up desired photography number of sheets and a criteria image assignment value 
by making the set point within the limit fluctuate by operating a cursor key 204 top or a down scrolling 
key by changing between setup of a criteria image to photography number of sheets, and operating the 
left or a rightward scrolling key. In addition, a setup of the above-mentioned criteria image is completed 
by pressing the decision key 203. 

[01 17] Next, in step S3, a user starts photography of the photographic subject image of at least two or 
more sheets. In addition, at this time, each image needs to be photoed so that parts may overlap 
mutually. And by supplying a photography signal to the main control section 14 from a shutter 113, 
whenever it photos an image, the increment of the counter built in the main control section 14 is carried 
out, and the number-of-sheets specification signal which shows the how many sheets the next 
photography is from this counter is supplied to the notice section 26. Since the signal which shows the 
above-mentioned criteria image assignment value from the criteria image setting section 20 is supplied 
to the register built in the notice section 26 here and the criteria image assignment value is stored in this 
register, In step S4, the notice section 26 compares the above-mentioned criteria image assignment value 
always stored in this register with the value which the above-mentioned number-of-sheets specification 
signal shows, and judges whether the image photoed shortly is what is used as a criteria image. 
[0118] And in the above-mentioned comparison, both value is in agreement, when the image which it is 
going to photo from now on is what is used as a criteria image, it progresses to step S5, and the notice 
section 26 notifies a user of it being photography of a criteria image. That is, at this time, from the notice 
section 26, a notice signal is supplied to an interface 16 through the main control section 14, and as 
shown in drawing 25 , according to this notice signal, the indicator 242 of photographic subject image 
35 width in a finder 241 is turned on with an interface 16. Therefore, it is recognized by the user at the 
time of photography whether it is photography of a criteria image easily. In addition, the notice in the 
above may be performed by displaying a predetermined text and a predetermined symbol on a display 
1 7 etc. 

[01 19] Next, although it progresses to step S6, when the image photoed next in step S4 is judged not to 
consider as a criteria image, it progresses to the direct step S6. And in this step S6, when it is judged 
whether photography (input of a photographic subject image) of a photographic subject was completed 
and it is judged that it does not end, it returns to step S3. On the other hand, when it is judged that it 
ends, it progresses to step S7. Here, it shall be made by pushing the switch with which a setup [ in / 
termination of photography is judged according to directions of a user, and / in these directions / the 
photography mode setting section 1 9 ] was changed to the normal mode, or was prepared for 
photography termination etc. 
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[0120] Next, at step S7, the focus is detected in a criteria image and the corresponding points which 
show the same part as the above-mentioned focus in the image which overlaps in these some criteria 
images [ at least ] are detected. Detection of such the focus and corresponding points is performed by the 
correspondence detecting element 21 shown in drawing 22 . In addition, the configuration and actuation 
of the this detecting element 21 are the same as that of the case of the gestalt 1 of the above-mentioned 
implementation. 

[0121] And while computing the parameter for amending the gate distortion of a criteria image in step 
S8, the image which instigated based on the above-mentioned parameter in step S9, and amended 
distortion is generated, and actuation is ended. In addition, although generation of the image which 
amended calculation of this parameter and gate distortion is performed by the image-distortion 
amendment section 22, the configuration and actuation of this image-distortion amendment section 22 
are the same as that of the case of the gestalt 1 of the above-mentioned implementation. 
[0122] As mentioned above, since a user is notified by the notice section 26 in the image photography 
by the user by setting up a criteria image assignment value before photography of whether the image 
photoed next is what is used as a criteria image according to the image processing system 8 concerning 
the gestalt 3 of this operation, a user can recognize photography of a criteria image easily. And a user 
can pay [ that the field of a request of a photographic subject especially goes into photographic 
coverage, and ] attention at the time of photography of a criteria image. Furthermore, the photography 
mistake of a criteria image can be decreased. 

[0123] In addition, it cannot be overemphasized that the technique which notifies a user of photography 
of a criteria image according to a setup of a criteria image assignment value by the notice section 26 can 
be applied also to the image processing system 6 concerning the gestalt 2 of the above-mentioned 
implementation. 

Although the image processing system concerning the gestalt of the [gestalt 4 of operation] above- 
mentioned implementation needed to photo the photographic subject twice [ at least ] from a different 
direction by moving the single optical system included in the image pick-up section, with the image 
processing system concerning the gestalt of this operation, two or more optical system is installed in the 
image pick-up section 41 side by side, and two or more sheets are once obtained by photography in the 
photographic subject image photoed from a different direction. 

[0124] Drawing 26 is drawing showing the configuration of the image processing system 40 concerning 
the gestalt 4 of operation of this invention. As shown in drawing 26 , although the image processing 
system 40 concerning the gestalt 4 of this operation has the same configuration as the image processing 
system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different from the image pick- 
up section 41 at a point including two optical system 1 1A and 1 IB. 

[0125] Actuation of the image processing system applied to the gestalt of this operation below is 
explained referring to the flow chart of drawing 27 . First, an image processing system 40 is started and 
photography mode is chosen. Here, the above-mentioned photography mode consists of the normal 
mode which takes the usual snapshot, and gate amendment mode which generates the image which 
amended the gate distortion of the photoed image. And selection in this photography mode is made 
when a user operates a photography mode setting key. In addition, as the photography mode setting 
section 19, the photography mode setting key 104 is formed in the body of an image processing system 
40. However, the photography mode setting section 19 may consist of hardware or software etc. which 
are formed separately from a body. 

[0126] And if it is judged whether it instigates in step SI and amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen as shown in drawing 27 , it will progress 
to step S10 and the snapshot of a desired photographic subject will be taken by the user. 
[0127] The criteria image as an object which instigates in step SI, will progress to step S2 if amendment 
mode is chosen, instigates to the criteria image setting section 20 on the other hand, and amends 
distortion is set up. As shown in drawing 28 , the above scrolling key 201 and the down scrolling key 
202, and the decision key 203 are contained in the criteria image setting section 20 here. Moreover, a 
user is asked for selection whether to use as the above-mentioned criteria image the image photoed by 
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which optical system 1 1 A and 1 IB while an overlay indication of the alphabetic character "criteria 
image setup" is given at a display 17. 

[0128] Then, by operating the above scrolling key 201 or the down scrolling key 202, a user moves the 
pointer shown with a triangle on a display 17, and specifies either the display the "camera 1" which 
chooses optical-system 1 1 A, or the display the "camera 2" which chooses optical-system 1 IB. And if 
the above-mentioned pointer presses the decision key 203 where one of optical system is specified, let 
the image photoed by the this specified optical system be the above-mentioned criteria image. Here, the 
setting information on the above-mentioned criteria image is supplied to the main control section 1 4 
from the criteria image setting section 20. 

[0129] Next, in step S3, a user photos a photographic subject. At this time, it is easily checked 
[ displaying only the image photoed by the optical system chosen at step S2 by control by the main 
control section 14, then ] by the user at a display 17 whether the image which is going to amend gate 
distortion includes the range of the request of a photographic subject. 

[0130] And in step S4, while the focus is detected within a criteria image, the corresponding points 
which show the same part as this focus within the image which has the field which overlapped the 
criteria image are detected. In addition, although detection of this focus and corresponding points is 
performed by the correspondence detecting element 21, the configuration and actuation of this 
correspondence detecting element 2 1 are the same as that of the case of the gestalt 1 of the above- 
mentioned implementation. 

[0131] Next, while computing the parameter for amending the gate distortion of a criteria image in step 
S5, the image which instigated based on this parameter in step S6, and amended distortion is generated, 
and actuation is ended. Although generation of the image which amended calculation of the above- 
mentioned parameter and gate distortion is performed by the image-distortion amendment section 22 
here, the configuration and actuation of this image-distortion amendment section 22 are the same as that 
of the case of the gestalt 1 of the above-mentioned implementation. 

[0132] As mentioned above, according to the image processing system 40 concerning the gestalt 4 of 
this operation, since the image pick-up section 4 1 is equipped with at least two optical system, a user 
can get the amendment image obtained with the image processing system applied to the gestalt of the 
above-mentioned implementation by one photography actuation. Moreover, if only the image photoed in 
the selected optical system is displayed on a display 17, since whether the image which is going to 
amend gate distortion includes the range of the request of a photographic subject can check easily by the 
user, a user can pay attention further to photography of a criteria image, and can decrease the possibility 
of a photography mistake. 

[0133] Moreover, it cannot be overemphasized that the image processing system 6 concerning the 
gestalt 2 of the above-mentioned implementation may be equipped with the image pick-up section 
which three or more optical system may be included in the image pick-up section 41 shown in drawing 
26 , and includes such two or more optical system. 

[0134] In addition, in the image processing system concerning the gestalt of implementation of all 
above, instead of photoing a photographic subject by the image pick-up section, the photographic 
subject image of two or more sheets stored in storages, such as storage, such as a hard disk, and CD- 
ROM, may be incorporated in enternal memory section 1 8 grade, and an amendment image may be 
generated using these photographic subject images. Moreover, that by which the photography mode 
setting section 1 9, the criteria image setting section 20, the correspondence detecting element 2 1 , the 
image-distortion amendment section 22, and image composition section 24 grade are stored in case with 
the another image pick-up sections 1 1 and 41, for example, a computer etc., is considered to be the 
gestalt of the above-mentioned implementation the same way. 

[0135] Moreover, although the correspondence detecting element 21 explained that corresponding points 
were detected by concentration matching by the correlation technique, you may carry out by another 
technique, such as a differentiation between space-time. Furthermore, in calculation of a gate distortion 
parameter, it is not restricted to the coordinate transformation shown in the above formulas (20), but 
other parameter calculation approaches may be applied. 
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In the gestalt of the [gestalt 5 of operation] above-mentioned implementation, although the coordinate 
transformation parameter was computed to the criteria image made into the object of an image 
processing and the distortion amendment image was generated using the interpolation operation, as for 
this criteria image, it is desirable that it is the image photoed by an image with little degradation of the 
image by the interpolation operation, i.e., the condition of having carried out the right pair to the 
photographic subject comparatively, including abundant paintings-and-calligraphic-works information. 
Then, the image processing system which performs the image-processing approach which chooses the 
optimal criteria image as below automatically, and this approach is explained. 
[0136] Drawing 29 is drawing showing the first example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 29 , although the 
image processing system 50 concerning the gestalt 5 of this operation has the same configuration as the 
image processing system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at 
the point equipped with the photographic subject field judging section 25 and the criteria image 
automatic selection section 27 instead of the criteria image setting section 20. 

[0137] Here, the photographic subject field judging section 25 and the criteria image automatic selection 
section 27 are connected to the main control section 14, respectively, and the outgoing end of the 
photographic subject field judging section 25 is connected to the criteria image automatic selection 
section 27. Moreover, the outgoing end of the criteria image automatic selection section 27 is connected 
to the image-distortion amendment section 22. 

[0138] In addition, also in the image processing system 50 concerning the gestalt 5 of this operation, as 
shown in drawing 4 , it can consider as the same configuration as the image processing system 1 
concerning the gestalt 1 of the above-mentioned implementation. 

[0139] It explains referring to the flow chart shown in drawing 30 in actuation of the image processing 
system 50 applied to the gestalt 5 of this operation below. The image processing system 50 applied to 
the gestalt 5 of this operation here is explained in detail focusing on difference below in order to operate 
like the image processing system 1 concerning the gestalt 1 of the above-mentioned implementation. 
[0140] First, in step SI, when it is judged whether gate amendment mode is chosen, and gate 
amendment mode is not chosen but the normal mode is chosen, it progresses to step S10, and a desired 
snapshot is taken by the user by this user. 

[0141] On the other hand, when it instigates by the user in step SI and amendment mode is chosen, it 
progresses to step S2. And in step S2, the photographic subject image two-times photography was 
carried out [ the image ] at least by the image pick-up section 1 1 is incorporated to a frame memory 15. 
In addition, at this time, each image needs to be photoed so that these some photographic subject images 
may overlap mutually. 

[0142] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
\ the photography termination by the user, it progresses to step S4. 

[0143] And in step S4, the focus is detected by the correspondence detecting element 21 in a 
photographic subject image, and the corresponding points which show the same part as the above- 
mentioned focus in the reference image which overlaps in these some photographic subject images [ at 
least ] are detected. Next, after detection of the focus and corresponding points is completed, the 
actuation which instigates below and amends distortion is started. And in step S5, the criteria image 
automatic selection section 27 makes first automatic selection of the criteria image which is instigated 
and is made into the object of distortion amendment. The criteria image automatic selection section 27 is 
explained in detail below. 

[0144] In addition, the parameter for instigating in step S6 to the criteria image chosen in step S5 like 
the image processing system 1 concerning the gestalt 1 of the above-mentioned implementation, and 
amending distortion is calculated, the image which amended the gate distortion of an image based on 
this parameter in step S7 is created, and actuation is ended. 

[0145] When amending gate distortion, it is suitable, if it crossed to the field where a photographic 
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subject side is the largest in two or more photoed images, and is reflected as the above-mentioned 
criteria image and an image with the abundant paintings-and-calligraphic-works amount of information 
which a user needs is chosen. Moreover, it is suitable if the image with the smallest possible angle (it is 
also called a "gate angle" to below) made to the photographic subject side of the image pick-up side at 
the time of photography is chosen. The reason is explained referring to drawing 3 1 . 
[0146] Although the image-distortion amendment section 22 shown in drawing 29 is instigated by 
performing coordinate transformation by the formula (25) as mentioned above and distortion 
amendment is performed, this amendment actuation changes according to the gate angle phi. Here, in 
order to give explanation an easy thing, the gate angle phi is restricted to the circumference of the y-axis, 
and it is assumed that the magnitude of the plane of projection 31-ed parallel to a photographic subject 
side is equal to the magnitude of the image pick-up side 32. 

[0147] As shown in drawing 3 1 (a), in the this [ angle / phi / gate ] coordinate transformation in the case 
of being comparatively small, the point near the left end of the image pick-up side 32 is projected by the 
variations shown by the vector which goes to Zero o to the plane of projection 31-ed. In addition, in the 
slash section in the plane of projection 31-ed, the resolution of a photographic subject image falls by 
such coordinate transformation. 

[0148] On the other hand, as shown in drawing 3 1 (b), it instigates in this coordinate transformation, and 
the point near the left end of the image pick-up side 32 is projected on the plane of projection 3 1-ed with 
a bigger variations than the variations angle phi is indicated to be to drawing 3 1 (a) when comparatively 
large so that the position vector to Zero o may be reduced more greatly. And it turns out that the field to 
which the resolution of a photographic subject image falls by this coordinate transformation becomes 
large as compared with the case of drawing 3 1 (a). 

[0149] It turns out that degradation of the resolution which followed, as mentioned above originated in 
the coordinate transformation by the image-distortion amendment section 22, so that the gate angle phi 
was small decreases. 

[0150] Moreover, although the photographic subject field judging section 25 shown in drawing 29 
detects the field which a photographic subject occupies in the photoed image For example, as indicated 
by reference "processing of an image and recognition" (Takeshi Akoin and Tomoharu Nagao 
collaboration, Shokodo) (a) How to cluster on an image like a field grown method or a field split plot 
experiment, (b) The field division approaches using the edge in images, such as the approach of 
clustering on feature spaces, such as field division by the histogram, and (c) profile line tracking, such as 
an approach and the (d) texture analysis, are applied. And the photographic subject field judging section 
25 chooses the image which photoed the large range of a photographic subject according to this 
judgment result as the above-mentioned criteria image. Consequently, an image including paintings-and- 
calligraphic-works information abundant as a criteria image can be chosen automatically. 
[0151] The image processing system applied to the gestalt 5 of operation of this invention here can also 
be considered as a configuration as shown in drawing 32 . That is, although the image processing system 
5 1 shown in drawing 32 has the same configuration as the image processing system 50 shown in 
drawing 29 , it is different at the point equipped with the straight-line-like pattern detecting element 28 
instead of the photographic subject field judging section 25. 

[0152] Generally, in the photographic subject side which makes a document the start, many straight- 
lines [, such as a character string and a ruled line, / which have an parallel relation mutually ]-like 
patterns exist. However, when the gate angle at the time of photography is large, it is projected on an 
image as a straight-line-like pattern in which the straight-line-like pattern which must originally be 
parallel has different sense. Therefore, by investigating dispersion in the sense of the straight-line-like 
pattern projected on the image, the size of this gate angle can be distinguished and an image with a small 
gate angle can be automatically chosen as a criteria image. 

[0153] Here, the straight- line-like pattern detecting element 28 shown in drawing 32 performs 
processing which detects a straight-line-like pattern in two or more photoed images. And an example of 
the detection approach of this straight- line-like pattern is explained below. 

[0154] First, an edge image is created by taking differential in these two or more images. Next, the 
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fragmentary edge point group in the created edge image is divided into a straight-line-like segment, and 
each straight-line-like segment is applied to the straight-line equation shown in the following equations 
(28). 

ax+by+c=0 (a2+b2=l) (28) 

here — the above-mentioned formula (28) — applying — it is made by applying a least square method 
using the point group which constitutes a straight-line-like segment. And termination of the reliance 
panel of the straight line to a straight- line-like segment searches for dispersion in being a parameter 
showing the linear sense (a, b). Thus, dispersion in the sense of the straight line in each image can be 
known by searching for dispersion in the parameter to two or more above-mentioned images of all (a, b). 
And an image with the smallest dispersion in the linear sense can be automatically chosen as a criteria 
image. 

[0155] Moreover, in automatic selection of a criteria image, the following approaches are also 
applicable. First, in two or more above-mentioned images, an edge image is created by the same 
approach as the above. Next, Hough conversion is performed to all the points in this edge image. Hough 
conversion is mathematical conversion used in order to detect a straight line from a fragmentary edge, 
and is the space constituted with the parameter of the formula expressing a line to detect here, and the 
approach of clustering is said. And more specifically, the sequence of points on an image are projected 
to the theta-rho space shown by the x axis, and angle theta and linear die-length rho to make. 
[0156] Drawing 33 is drawing explaining Hough conversion to the theta-rho space shown in drawing 33 
(b) from the image space shown in drawing 33 (a). Each point P1-P3 on an image is changed into 
drawing 33 by Hough conversion at the curves L1-L3 which correspond, respectively so that it may be 
shown. And although the point CP which the locus of this curve is concentrating in theta-rho space will 
arise if such conversion is performed about the all points on an image, this point CP is equivalent to the 
straight line which passes along many edge points in an edge image. If the coordinate of this point CP is 
set to (theta, rho), it will mean that the straight line corresponding to the following formula (29) was 
detected here. 

Rho=xcostheta+ysintheta (29) 

And dispersion in the sense of the straight line in each image can be known by extracting many points 
CP which this curve concentrates to two or more above-mentioned images of all, and searching for 
dispersion in theta in these points. Thus, what is necessary is just to choose automatically an image with 
the smallest dispersion in the linear sense as a criteria image. 

[0157] Drawing 34 is drawing showing the third example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 34 , the image 
processing system concerning the gestalt 5 of this operation is good also as connecting the criteria image 
automatic selection section 27 to the correspondence detecting element 21, without having the 
photographic subject field judging section 25 shown in drawing 29 , and the straight-line-like pattern 
detecting element 28 shown in drawing 32 . 

[0158] The group of the focus detected by the detecting element 21 corresponding to the above and 
corresponding points is used for count of the parameter which amends the projective-transformation 
matrix B, i.e., gate distortion. At this time, generally, and it is computed with so sufficient that they are 
distributing broadly a precision. [ this parameter ] [ the group of the above-mentioned focus and 
corresponding points ] Then, the distribution in the groups of the focus detected by the correspondence 
detecting element 21 and corresponding points and those images may be investigated to two or more 
photoed images, and the image those values of whose are maxes may be automatically chosen as a 
criteria image. 

[0159] Moreover, in the correspondence detection based on the above-mentioned correlation technique, 
much focus and corresponding points are detected, so that there are many patterns which generally have 
the description in an image. And since possibility that many paintings-and-calligraphic-works 
information useful for a user in an image is included is high,if the focus and corresponding points make 
automatic selection of the abundant images as a criteria image, that there are many characteristic 
patterns can instigate an image including required paintings-and-calligraphic- works information, it can 
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set it as the object of distortion amendment, and is suitable. 

[0160] Drawing 3 5 is drawing showing the fourth example of a configuration in the image processing 
system concerning the gestalt 5 of operation of this invention. As shown in drawing 35 , the image 
processing system concerning the gestalt 5 of this operation is good also as having the flat-surface 
measurement section 29 instead of the photographic subject field judging section 25 shown in drawing 
29 , and the straight-line-like pattern detecting element 28 shown in drawing 32 . 
[0161] Here, the flat-surface measurement section 29 measures the sense of the photographic subject 
over the image pick-up section 1 1 at the time of photography of each image. Drawing 36 is drawing 
showing the example of a configuration of the flat- surface measurement section 29 shown in drawing 
35 . As shown in drawing 36 , the flat-surface measurement section 29 is equipped with the spot light 
source 271, a photo detector 272, the three-dimension coordinate calculation section 273, and the flat- 
surface calculation section 274, and the spot light source 271 contains light source 271a which consists 
of light emitting diode, semiconductor laser, etc., scan mirror 271b, such as a polygon mirror, and 
mechanical-component 271c which controls a motion of scan mirror 271b. 
[0162] Here, in the above-mentioned spot light source 271, scan mirror 271b is controlled by 
mechanical-component 271c so that the spot light generated by light source 271a hits the photographic 
subject side PL. Moreover, a photo detector 272 is constituted by optoelectric transducers by which the 
location to the spot light source 27 1 was installed in the location measured beforehand, such as PSD 
(Position sensitive detector) and CCD, and detects the sense of the reflected light from the photographic 
subject side PL. In addition, the optoelectric transducer 114 contained in the image pick-up section 1 1 
may be used as a photo detector 272 in the above. 

[0163] Moreover, the three-dimension coordinate calculation section 273 computes the three-dimension 
coordinate (X, Y, Z) of the photographic subject side PL on the basis of an image processing system 53 
by using the principle of triangulation according to the sense of spot light which the spot light source 
271 irradiated, and the physical relationship of the spot light source 271 and a photo detector 272 and 
the sense of the reflected light which the photo detector 272 detected. And the flat-surface calculation 
section 274 presumes a flat-surface equation using the three-dimension coordinate of three or more 
points which is not on the same straight line computed by the three-dimension coordinate calculation 
section 273. For example, it is the flat-surface equation for which it asks aX+bY+cZ+d=0 (a2+b2+c2= 1 , 
c> 0) (30) 

It sets and four parameters (a, b, c, d) in the above-mentioned flat-surface equation are calculated with a 
least square method using the three-dimension coordinate of three or more points. Consequently, the 
above-mentioned gate angle phi is calculated by the degree type (3 1). 
Phi=eos- lc(31) 

Therefore, what is necessary is to measure the sense of a photographic subject side to two or more 
photoed images of all, and just to choose automatically the image the gate angle phi of whose is min as a 
criteria image. 

[0164] In addition, as described in explanation of the image-distortion amendment section 22, it can ask 
for the sense of this photographic subject also by using the group of the focus and corresponding points 
which were detected by the correspondence detecting element 2 1 . Therefore, the image processing 
system concerning the gestalt 5 of this operation can once perform a formula (14) thru/or a formula (18) 
in the image-distortion amendment section 22, can ask for the sense of a photographic subject, and can 
also consider the obtained result as the configuration of outputting to the criteria image automatic 
selection section 27. 

As shown below in [the gestalt 6 of operation] at drawing 18 , the gestalt of the operation on condition 
of the case where the static image in the same photographic subject side PL is photoed from two or more 
directions so that it may overlap in some images, respectively is explained. In addition, as shown in 
drawing 18 here, by Hazama of Image imj and Image im 0+1) (1 < == j<=K -1) which were obtained by 
taking a photograph in the direction which Image imj (1 <=j<=K) is obtained, for example, adjoins each 
other like an image iml and an image im2, there shall be a duplication field by taking a photograph from 
Direction dj, respectively. And in the image processing system concerning the gestalt 6 of this operation, 
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other images are stuck and a synthetic image is generated so that it may have consistency in the gap or 
one image chosen as a criteria image. 

[0165] Drawing 37 is drawing showing the configuration of the image processing system 60 concerning 
the gestalt 6 of operation of this invention. As shown in drawing 37 , although the image processing 
system 60 concerning the gestalt 6 of this operation has the same configuration as the image processing 
system 6 concerning the gestalt 2 of operation shown in drawing 19 , it is different at the point equipped 
with the criteria image automatic selection section 27 instead of the criteria image setting section 20. 
[0166] The image processing system 60 concerning the gestalt 6 of this operation which has the above 
configurations is explained focusing on difference, referring to the flow chart shown in drawing 38 in 
actuation of the image processing system 60 applied to the gestalt 6 of this operation below, although it 
operates like the image processing system 6 concerning the gestalt 2 of operation. 
[0167] First, in step SI, when it is judged whether image composition mode is chosen, and image 
composition mode is not chosen but the normal mode is chosen, it progresses to step S10, and a desired 
snapshot is taken by the user by this user. 

[0168] On the other hand, when image composition mode is chosen by the user in step SI, it progresses 
to step S2. And in step S2, two or more photographic subject images photoed by the image pick-up 
section 1 1 are incorporated to a frame memory 15. In addition, at this time, each image needs to be 
photoed so that these some photographic subject images may overlap mutually. 

[0169] Next, it is judged whether the input of a photographic subject image was completed in step S3, 
when having not ended, it returns to step S2, and the input of a photographic subject image is continued 
further. On the other hand, when the input of a photographic subject image is ended with directions of 
the photography termination by the user, it progresses to step S4. 

[0170] And in step S4, the focus is detected by the correspondence detecting element 21 between Image 
imj and Image im (j+1 X and the corresponding points which show the same part as the above-mentioned 
focus in the image im (j+1) which overlaps in these some images [ at least ] imj are detected. 
[0171] Next, after detection of this focus and these corresponding points is completed, the actuation 
which generates the synthetic image which stuck the image of two or more sheets on below is started. 
And in step S5, the criteria image automatic selection section 27 chooses automatically first the criteria 
image made into criteria in this composition. Here, the criteria image automatic selection section 27 
chooses automatically the image with the above-mentioned smallest possible gate angle as a criteria 
image. And the synthetic small image of gate distortion can be obtained by performing such selection. 
[0172] And in step S6, the projective-transformation matrix for performing composition on the basis of 
the criteria image chosen in step S5 is computed, a synthetic image is generated using this projective- 
transformation matrix in step S7, and actuation is ended. 

[Gestalt 7 of operation] drawing 39 is drawing showing the configuration of the image processing 
system concerning the gestalt 7 of operation of this invention. As shown in drawing 39 , although the 
image processing system 70 concerning the gestalt 7 of this operation has the same configuration as the 
image processing system 1 concerning the gestalt 1 of operation shown in drawing 3 , it is different at 
the point further equipped with the criteria image automatic selection section 27 and the change section 
45. 

[0173] Here, the criteria image automatic selection section 27 is controlled by the main control section 
14. Moreover, as for the change section 45, an outgoing end is connected to the image-distortion 
amendment section 22 while the input edge is connected to the criteria image setting section 20 and the 
criteria image automatic selection section 27. 

[0174] Moreover, the change section 45 changes the selection approach of a criteria image, and as 
shown in drawing 40 , it contains the above scrolling key 201, the down scrolling key 202, and the 
decision key 203. Here, when a user pushes alternatively the menu screen key (not shown) displayed on 
a display 17, the screen where an overlay indication of the alphabetic character "selection of a criteria 
image" as shown in drawing 40 was given is displayed on a display 17. 

[0175] And when the above scrolling key 201 or the down scrolling key 202 is operated by this user, the 
cursor of the triangle shown in drawing 40 carries out vertical migration. At this time, if the decision key 
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203 is pressed in the condition that this cursor has pointed out the alphabetic character of "AUTO", the 
criteria image automatic selection section 27 as which choosing a criteria image automatically was 
determined and it was indicated to be to drawing 39 will be activated alternatively. On the other hand, if 
the decision key 203 is pressed in the condition that this cursor has pointed out the alphabetic character 
of "MANUAL", the criteria image setting section 20 as which setting up a criteria image with hand 
control was determined, and it was indicated to be to drawing 39 will be activated alternatively. 
[0176] Therefore, according to the image processing system concerning the gestalt 7 of this operation, 
by the change by the change section 45, since either the criteria image setting section 20 or the criteria 
image automatic selection section 27 is alternatively connectable with the image-distortion amendment 
section 22, a user can choose automatic or manual either as arbitration as the selection approach of a 
criteria image. 

[0177] In addition, also in the gestalt of which operation mentioned above, the image-processing 
approach concerning the gestalt of this operation can be described as a computer program. And as 
shown in drawing 4 1 , the above-mentioned image processing is easily realizable by equipping an image 
processing system 1 with the record medium 301 which stored this program, and making an image 
processing system 1 perform this program. 

[0178] Moreover, as shown in drawing 42 , the above-mentioned image processing is realizable also by 
equipping a personal computer (personal computer) PC with CD-ROM302 which stored this program, 
and performing this program with a personal computer PC. In addition, as a record medium with which 
a personal computer PC is equipped and which stores this program, it is not restricted to above- 
mentioned CD-ROM302, for example, it cannot be overemphasized that you may be DVD-ROM etc. 
[0179] An example in case this program execution realizes the above-mentioned image-processing 
approach below is explained. In this case, an image processing system 1 and a personal computer PC are 
incorporated to the above-mentioned signal-processing section 12 through the various interfaces in 
which the photographic subject image of two or more sheets stored in record media, such as storage, 
such as built-in memory and a hard disk, and CD-ROM, was carried by the computer. 
[0180] Moreover, a setup in the photography mode in the above is carried out by carrying out the mouse 
click of the icon displayed on push or a screen in the predetermined key in the keyboard of a personal 
computer PC. The image by which infanticide was carried out [ above-mentioned ] on the other hand by 
operating on a curtailed schedule and displaying two or more inputted images on a screen, pushing the 
vertical cursor key of the keyboard used for a calculating machine, or clicking on an icon with a mouse 
in choosing a criteria image with hand control etc. is chosen. And if a line feed key is pressed where a 
desired image is chosen, this image will be set up as a criteria image. 

[0181] Moreover, about the focal distance f of the optical system which constitutes the image pick-up 
section 1 1 , the focal distance of this optical system is measured beforehand, and it records on the interior 
of this record medium. And a user chooses the focal distance of the optical system used on the screen on 
the occasion of actual photography. Here, this focal distance is also recordable as header information. 
That is, when using an Exif format, for example as image data, the focal distance at the time of 
photography can be recorded as the header information. And the image processing system concerning 
the gestalt of this operation can obtain this focal distance by reading this header information. In addition, 
when the above approaches cannot be taken, it is good to display a dialog box etc. on screens, such as a 
personal computer PC, and to make a user do the manual entry of the direct focal distance into this 
dialog box. 

[0182] Moreover, it can treat about direction of the photographic subject in the above as well as the 
above-mentioned focal distance. That is, the sense of the photographic subject measured by the flat- 
surface measurement section 29 or the image-distortion amendment section 22 is beforehand recorded as 
header information in an image data file, and the image processing system concerning the gestalt of this 
operation is good also as obtaining the sense of this photographic subject by reading this header 
information. 

[0183] Although the field for recording the sense of a photographic subject does not exist in using an 
Exif format as this time, for example, image data, the sense of a photographic subject is recordable on 
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the field which the manufacturer called Maker Note can use freely. 

[0184] Moreover, when the above approaches cannot be taken, it is good to display a dialog box etc. on 
screens, such as a personal computer PC, and to make it make a user input the sense of a direct 
photographic subject like the case of a focal distance. 

[0185] In addition, the gestalt of operation of above-mentioned this invention can be applied to image 
processings using a digital still camera or a digital camcorder, such as a space information input and 
lamination composition of a division image, and can be applied also to the image device of a non- 
contact handy scanner or others. 

[Effect of the Invention] Since the image optimal in order to obtain a proper image by choosing the 
object which amends distortion out of two or more images in the image-processing approach which 
amends distortion of the image photoed from two or more directions can be made applicable to 
amendment like **** according to this invention so that at least a part may overlap to a photographic 
subject, this more accurate amendment is realizable. 

[0186] Moreover, distortion chooses fewest images from two or more images, and if the image which 
amended distortion is compounded with the this chosen image, a more proper synthetic image can be 
obtained. 

[0187] Here, if the object of amendment is automatically chosen according to the size of the field which 
a photographic subject occupies into an image, since the paintings-and-calligraphic-works amount of 
information needed can amend distortion of the most abundant images automatically, a certainly proper 
image can be obtained. 

[0188] Moreover, if the object of amendment is automatically chosen according to the sense of the 
straight-line-like pattern detected in an image, since the image photoed from the location which carried 
out the right pair to the photographic subject mostly can be automatically made applicable to 
amendment, a proper image with high resolution can be obtained. 

[0189] Moreover, if the object of amendment is automatically chosen according to the specified 
correspondence relation, since high amendment of precision can be performed certainly, the 
dependability of an image processing can be raised. 

[0190] Moreover, if the object of amendment is automatically chosen according to the sense of the 
photographic subject detected for every photography, since the image photoed from the location which 
carried out the right pair mostly to the photographic subject can be automatically made applicable to 
amendment, a proper image with high resolution can be obtained. 

[0191] Moreover, since according to the image processing system equipped with a notice means to 
notify a user of becoming the image made into the criteria at the time of the image photoed next 
amending distortion attention is called in the photography of an image made into criteria to a user in 
case distortion is amended, the photography mistake by a user's inattention etc. is avoided and 
dependability of operation and the quality of an amendment image can be raised. 
[0192] Moreover, according to the image processing system equipped with two or more optical means 
which photo a photographic subject to coincidence, and a selection means to choose the image made 
into the object of amendment among the images photoed by two or more optical means Since two or 
more photographic subject images once photoed by photography from two or more directions can be 
obtained, while actuation is made simple by decreasing the count of photography needed in order to 
amend distortion, the image with which distortion amendment was carried out more simply can be 
obtained. 



[Translation done.] 
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1 9 ebb*- K*Hj£-r s TdtxDffife*:- kk£*- 

1 0 4 g^gp 1 7£&£ftfcBB£±73lSj'\72n 
— ;UT3fc&<D±7j[RjX£a-;l/+-2 0 1 i:, gjfcgB 

1 7KW£nfcB^£T73l^7^a-;l/?-Sfc&<DT 
73(rJX^D-;1/+-2 0 2t, i&£*-2 0 3 40 

tr. 

[0 0 3 4] fiTFCtelT, ±E©J: 5 
BB$aagB©ftf££, 04 t05{c^^nfc7n-f- 
■^-hi:?r#^Lo-3llS^-rs 0 SfBfflfc, BB>W 
7fl01 *«9#*TH«U&1!»B1 *S»U 

X-*->y7¥K£»fgf sa^-Fi:, JtiJLfcBfS© 
fete £ IE L fcBB*±Bf S fete !? ?SjE€— F 

<£D£So ^UT, c©BB*-F©B8?tt, a— «ftf 
BB*-FBJ£+— 1 0 4*#fttSCbK.£*)1i:Ztl 50 
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10 

S 0 &te}115*- F^SaP 1 9 b LT, IWiSi 1 

(D^mam&t- fr^*- i o 4 tfRtt&ns. b 

U BB*-K»g»l 9t3u *(*fc(4B!HitKt*6n 
SM-F7x7XtiV7h7x7^FT-«^LTtav\ 
[0 0 3 5] ^UTBSfcljj^ftSJc-Mc, 7x>y7S 
1 (cte^TfetetiffiiEt- K*»?-r 5 *»S*»W»f* 

n, fete t) be*- mmsizriT&n*:- vi/ma 

ftSfc, Xr77S 1 «ftJ:DBTa©*Pf 
ft©7:>- -y 7?IAl»?n5= 

[003 6] -73 > 7-r -y 7 S 1 Ete^Tfete •? MJE€- 
-KtfBBSnSk, 7f-y7S 2Ni€r. ^LT, a 
-If «£BgB 1 1 K «fc 5 P L < t fcr ft 
Ji^U BB&agfilfc{*&¥fl:Btf&t)&3:nS. 
&te, tOfc*Sv^:«^B©-»tfB4t)£3J:$ 
{{^©BBtfBf&^ftS&BtffeSo 

[0 0 3 7] 7.x -y 7 S 3 Ete^T$¥#<li©A 

7J*^7 U/tft»g*»AWRIK *I7 LTV*v>«fcfc 
«7x-y7S 2'\M0x £&E$¥ft&©A;Wiltit£ 
nSo -73, a— <ffc<fc*ffl&«7©JBjj*fcJ:*>#5ft 
$© A7J £H7 L fcB^K fi, Xr-y7s 4"\jttr„ -4 
fc\ iBBBiSttBB*- KRj£*- 1 0 4 *t 5— tfS¥ 
UTam^-FfC^JDg^SCfctciO^^tiSfS, A 
K^7^JI^S rctb<DX<{ «y ?-*8!BfcRttTfe <t V> 0 

[0 0 3 8] $fc, Ji^4'fi:tev^T(iS6fc^ns<i; 

[0 0 3 9] fLT, ±IB©±5fCfflKAWrSi:, 

7r-y7s 4WT{ctei/^Tfetet)S^?iiE-rsKimi: 
AS^, S"f7-r>y7S 4fctev>Tif©B#Ofetet»^ 

*^iE-rs^*s^bTKB»^pas-rso c 

0fctev>TfTt>tiSo t^Tlctev>T, g*PB#l35£a$ 2 

0 o«sRiflBft t < mm? 3 o 

[0 0 4 0] 07 U\ H3K^n^^PBBB£ff 2 
0cDbl'77h^-riaT*feSo 07t^$nSJ:9 
fC, g*PB#ia£g|$2 0{±±73lRl7^P-;l/+-2 0 1 
i:, T73[Rl7^P-;l/+-2 0 2Slf^*-2 0 3S 
■StSo fit, 0 7t^?ns<J;3t7-r-y7S 3lcte 

1 7tctev>T rsipH^e^j t^^^-z'in' 

So 

[00 4 1] -fSta— tffcfei?, ±7J[rI7?o-;1/* 
-2 0 1 J ?T73|Sl7i'D-;l/^-2 0 2*«ff*tU ± 
E7-r-y7S 2fctev>rBK$nfcBB^B^0§^ 
O-og^^nSo 4te> ±73|qJ7.^n-;l/+-2 0 IS 

jesnfcBBtf^artt, ^/5:T73[rJ7^p-;1/+-2 
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0 2*nt£ fcCJCtK ftTV>5B#© 1 "D 



[0 0 4 2] *LT±E»*B«i:LTH\ 

[0 0 4 3] £<LT\ «P¥fl««i:±Kl«»ft fc*SEH 
-f5*f#U Tfcfo-5S*B»i:LT©B:&B&*£Klto 



<fc 5 *f&ft*^a-f sB&aaags 2 ©*BfiSc£^-f bt* 
o <DKt> *) Ktos#m$&i3m 2 3 turn*. ns. 

[0 0 4 4] ^LT, ±eWPfl*«Sa£»2 3fi, it 
&LfcB#£fc^T^tt#£i&SBi£*fcfflT*$!lS 20 

(a) B«raa&*flH«#Mffi©*5fcB^?*7X 

(c) HIPSjiie):^HOB^+<OX-yv , ^ffl^^73^ 

(d) fj'Xft^, fc£©B&#B;£i£tf3iiffl2n 

[0 0 4 5] £©<fc?fct£¥#Bi£ifc£gP2 3Tf#5ft 
fctt¥#0fiW*&a*-r7*-*U\ B^5^ffiIEgP2 2^\ 
«S&^n, ^fiW«k£V«#B#^*ffiIEa3 2 2 



[0 0 4 6] &*>\ J|JU:©J:54a»«K»PB«*a 



(7) 2 0 0 1 - 2 5 0 1 1 4 

12 

* [00 4 7] ^Wcx-ry^S 5^31^ ±!B©J:5£ft 

JpB$j©4>&< £&-SP£i3^T»g-f SB® (feTFC 
fcv^T, r#BiBISJ tt>W£o ) ©"fKfc^Tf^fE 

tb»2 i©«fi!tRt;i)<f ; &i¥t<^-r5o 

[0 0 4 8] *f jS&tfJgp 2 1 tt. ±15© <fc 5 KSWca 

[0049] B 9 (±, B 3 K*£*lfc»J&t*tH» 2 1 © 
352 1 tt7WW*iJ 1 5KaH**tifcWfUldBi» 

21 it, mm&ffl£®2 1 iatf7u-i»ytu 1 5 

ICimS tlfcfflHWm 2 1'2 B3 

tc^tifc 7 u— i»**y 1 sic& s*H{fc£*sgB 

[0 0 5 0] C CTttlUnR}aV2 1 1 tt» £¥HfiHC 

i:TS (2N+1) x (2P+1) 
fcJ*1$fb£«D{&BH\ ft (corner) ©£ 5 £B#©*1 

[0051] ttcmmmw^2 1 2«, sspB^tst? 

30 ttfsu cti«»£A2:%ei"«. ffiBiStSfc 
J; 5 7p 7 * v y ^ «fc 0 Jt(SjS*«i at * -B K 

ov>t, bi o%&mL-D^>mm?%o 

[0 0 5 2] Bl 0fc3**fiSJ:3fc» (2N+1) x 
(2P+1) «©afifcrt*->ft»Sa.5fi§»fc2 1 5, 
2 1 6©7a-y ^v-y ^->^*f;:fel,>T, g*PB#7F*3© 
BfllCxio . y.o ) £W-f3 i#B©1#t!^i2 1 3 
t, #B8Bf&9F*i©£8l (x.o +dxi , y.o +d 
y. ) £#"f3*fj£j&2 1 7©tB5taMfflS. tt, 

40 [00 53] 

mil 



±IHS (1) lCfel,^TI. (x. y) tiStpB^ Bfi*^r. / I. (x, y) ttS*|SB#7tCfc 

1V>W&& (x, y) CfcltSBftt^U I lt?)ffiig^2 1 5rt©ffi<l^ (x, y) Z^fotLfc 

, (x. y) W:#MB»9©£SU5 (x. y) tcfclt?> 50 (2N+1) x (2 P+ 1 ) B©^^->fcfelJS¥^ 
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«fi*^U/Ir (x. y) tt#!$W&9tc*5W-5*B 
I9&2 1 6F*gO^S^ (x, y) 4N>&fcUfc (2N + 

i) x (2P+D mw^-yizisifzw-izmmz^ 

[0 0 5 4] fLT, ±§BS (1) KJ:?>#1#ftj£2 1 
■ffilJLh-e* £ £ <fc (5, #^H#9tcfc 

ttsftj£j£2 1 7 3b^festi5 0 ^fes tastasafits. 

[0 0 5 5] il©<k9tcLr> 4#$;S£#JSj&0&ffl# 
&7-r3^ H5(c^^n/cXT-<y^S 6tfcv^TS* 

it{c, Xf'^S 7£*5V^T^A^-#£gt;:B#£> 
IEBSU tt>V^. 

[0 0 5 6] f LT, ±E£»«jE/<7*-*©ff-*ft 
tfil**SiEil«W)^tt, H«g*ffiiEgi5 2 2 fc J: 5 $ 

[00 57] ■&£2rifjE$ 2 2 W\ ttJS&WSP 2 1 

%MiE-r^c ^-utx ucoiB^s^ffiiEgi52 2 <ommt 
mi uc^snso 01 iic^*i5<t?K:, w&&& 

ffiiESP 2 2 (i 3 #JciSt*gl5 2 2 1 k>^ 2 
2 2RtfffiS^SI52 2 3£#t? 0 CCX\ 3&nMW 

gP2 2 1 ttsjs^msp 2 1 twm»i&£8ii2 oic&m 

2 ft, /Ut— 2 2 It 3 WtffiM SP 2 2 1 K 

a»«ft«. fia3»»2 2 3tt/<5*-**a 

gP2 2 2£7U-A**'J 1 5 &tf£!PH& K£# 2 0 
[0 0 5 8] WTfcfcVT, BftS»fll£ffi2 2€>ISft 

$n5<t9tC, x|fifcMLT(4B#®2 2 4«D^|r1^^ 
HE, ylStcMLT{±H#®2 2 4cDT[r1#^E, ft«73 
6K0 zfflfcM LTttSMR* 1 1 O3t^>t>^-pa5«B^0 
a»5H«l®2 2 4'\iajfr5lRl**IE, MM£¥9RoiRjAE 
glfif ££*l5't'£>ft&t'r;l' (perspectiveprojectio 

_ &«x, » 6 t .y, + bt 



(8) #1^2 0 0 1 -2 5 0 1 1 4 

14 

*n model) t^tX^^mthXm^t^o 

[0 0 5 9] 01 1 fc^tlfc 3 ^TCiltggP 2 2 1 tt, 
±EWfb62 1 3 fc*tJ&jS2 1 7 fcQHftiCj; t>, UCF 

®3^7c^7^-^?:scm-r-5o -r&fr-^ 
fiJi^^stctrc#MH®Jt^cfeitsa^gpi i 

10 T, Cft5 30©3^7C^7^-^(R, t. n}£j£i& 
[0 0 6 0] t%;t>*>m~<Dl5'&tLZ, 80J^±©# 

[006 1] %tcm-<Djj&t LT, 4 0W±©1t^ 
t#JW&<Dffl<J: t), WKSIftfTlU (holography matri 
x) *fit*U 1i5tife«»«SWf5«JteJ:»)*aiiiaW 

[0 0 6 2] llT% JtSES— ©£i£W\ iaffl©ai)i£ 
flcKStfi-?* 0 x S^iflgK <£ 9 ±12^7 ^ - ^ { R , 

30 lRl$*»tf}-rSt»OT*S5 0 ^LT, 3^7cSISgP2 2 

[0 0 6 3] $-f»»^WfWO»lti#JB%BL<KW 
tt, 01 4te^n5<fc9{c x S*PB»7fc^ofcg^ 

40 (x, . y, ) i:#B8B«KfcltS^ (x r , y, ) t 

[0 0 6 4] 
[»2] 



(2) 



^■bT, ±EOS (2) tC*J(t5 8O0*ftl^bi ~b 50 
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[00 6 5] 



fa 
b< 

lb, b, 1 



b s b t 



(9) 



* * [&3] 
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(3) 



tmiztLTZtib, cft*M&mmmtw&o c *s (2) ^Atrb. ~b. o^^tontfiv^v 

# (x»i , y ., ) -e&zttm&t, whmuc&h- las&LKwi?, in-riz^.rm^fmw^m^xm 

Zmmt (x,. , y,i ) T?£3ftjSj& ( i = 1 , 10 <ili:{cfc5o 

• -, N; N>4) t<Dffl^4ffll^±fflV>§ 0 CCT\ & [0 0 6 6] 
m (x.« , y.. _) (x,, . y,, ) £±fH* [S4] 



E 



-,.)• 



min 



(4) 



f LT, ±fBS (4) ttOT©J:5{;«£ft3, 
[0 0 6 7] 



★ [&5] 



(5) 



nun. 



±IBS (5) ©£j2£bi ~b» T'^n^tl^^LT [0 0 6 8] ^V">Ts l>f ^^fT5"J B <t 3#7C/1v^ 

^f?>nfu^lp^^Dffl^o^c^^t^^9ifi)*^ff^r^fJffl•r -*m. t, nj^fes^m^sKB^-r^o tmwmv 

b, ~b 8 «aa:-^ss^< z.t\z&K>^ &m<?h>i<nz 

nznzo M^mmmBamnmifz [o o 6 9] 

a 

b (a t + b 2 + c I =l,c>6) 

c 



« = 



(6) 



m%m*mmt Ltdm-wm^m^. ♦ [0070] 

(«, r)+ = 0 



(7) 



CCT\ I d I im&frbfa^-WmP Lf T'©g£gI%;J; * [0 0 7 1] 



U r = [x y z] T irfcfrftSo Sftiu&iE 
f£JS^T, S (2) «^OJ;9fc«$B^tx5o 

H a x, + H n y, + ff»f 



40 [$8] 



(8) 



(8) It 
[0 0 7 2] 




tmj&ztiZo fit, 

[0 0 7 3] 



Z'&Zo C©B$> 3 (9) <DftMHW7*-*{R. 

t, n, d} tcommt. .fe^JcdtCftSo * 

fit, s * ajaR-p*^ 'imu<D&mmtex'r-)Hg 

<Dgifi£WbTt^So ^ (3) (DM&mmm 



* CSi o] 



* [0 0 7 4] 
[ISll] 



"1 0 0" 




"1 


0 




1' 


0 I 0 


B 


0 


I 


0 




0 0 / 




0 


0 







★ fctfT'tSo 
[0 0 7 5] 

mi 2] 
LA A i 



(10) 



0 0 



(12) 



n, d}^»-5?m^-r 0 fiu, WMvmtoim 

[0 0 7 6] 
[&1 3] 

lltll=l (13) 

tfafcfZo '^*—5<{R, t. n, d}©» 

(i) S (9) ©?fMH®«8Rfc»S&£»**»tt 
T, d e t [H] = 1 fc*SJ:5K"r5. (ii) Sfftfr 
JIJHH* OitMo, 2 . oi ', 03 2 £U MJStV 



z >Oa >0£"fS o (iii) 0i =02 =03 5:5 

[0 0 7 7] 
OR 14] 

30 t = 0, R= H (14) 

(iv) WMtimw^*-* <n, d} tt, #©«fc5fC 
[0 0 7 8] 

cai 5] 



Vcr, -cr, 



[0 0 7 9] 



♦ ♦ [Si 6] 



rf = - 



(15) 



cr, - cr. 



(16) 



fc, (v) ^ffiMJsafi^^ t * a ic^s 

So * 



* [0 0 8 0] 
[»1 7] 



' " ~ ^~^ tl> -*nR (a*ra«) 



(17) 



(11) 



19 

[0 0 8 1] 
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20 



* mi 8] 



cr»L (/j. t)+d J 



1 1 ©«u£ee« f *ffijB*-*#, jr^es f offlttja® 



10 



(fc©*ltf5TO#K»T*fcftfcr\ ±ffE 3 #tc/ < 5 * ~ $ W 

[0 0 8 2] 01 1 KSSftft^7^- *WtfJSB 

2 2 2 $¥&35%jii?LfcB#©jf#gi5 1 1 t 3#7C 



(18) 



K, 3«7afW*2 2 1 jBWW LfcSIWlCffi 
ffi£$&ig®3 1 £ LT, &&9£&«&-rs£i:#fil 

If. iS^S3 0±©£P 1 liffiS^B3 1 ±©£P 2lc 

7 * ~ * annum zmnt % . 

[0 0 8 3] st\ 01 6fc^n5J:5»c, gBgg 
&3 3fc*tt5 ztt*tt&KS3 1 OJMK&ttt* 

[0 0 8 4] 
[»1 9] 





"0" 




a 


J? 


0 




b 




1 

• J 







(19) 



*lt, ±es o) £»/c-fMT5UR' tt^a#ft 



★ [0 0 8 5] 
[»2 0] 



/?' = /?;/?; = 



(20) 



fiU dCT'R' , fcR* y H*ftf f ft#©J:3K3*S & [0 0 8 6] 

_ ^ DR2 1] 

'10 0 
0 cosa — sina 
0 sina cosa 



K = 



cos/3 0 sin/7 

0 1 0 
-sin/? 0 cos/7 



(21) 



Ctttt, 01 7£^£ft5«fc3C, trF©«?T*= 
(xyziSS) 3 3£lelte£-frT, x' y* z' 

HUcmrr«c2:icffi3-r«. a) swans 3 

3*yW©@0fc^W-/J|g(E*«5o ^LT, C©@|g 



a = sin 



-i 



(-6) 



♦ (li) 8S£ft$$x l fft©|slt)t«:a^ftlHlfe-rSo 
[0 0 8 7] CCT\ 5* (1 9) £3 (2 0) *JBV^5 

[0 0 8 8] 
B&2 2] 

(22) 



(12) 



21 



ZLT, c©«fc3E**&ftfciaiEa*S (2 0) RV 
S (2 1) JCftA-rsCttiD, fT?"JR' fc-BEffi 

[0089] #e, wmw3 o±offi««aa6»ffi3 1 
±fcjfflas» - rSo r &fc>s, 0 1 5 e*ji,>t, b^b 
3 0 ©£ p 1 k # js-r 5 3 $&t* ? h n p t ft t * 



J 

y, 

./J 
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* ttcssK®3 1 t&mtz&p z*. mmmmkcom 

MttZo fit, SBSB$3 35-aaptLfc^P 1 

{c»is-r§ 3^7c^^ hfrpit, jfcs-eassn-So 

[0 0 9 0] 
[S2 3] 



(*>o) 



(24) 



tt, £¥B£££H$<DSB£B?£ 3 3 ££9E LTj^P 



P' = 



X 
Y 
Z 



= JR" l P = 



ax.+by. + c/ 



[009 1] 
BR2 4] 

*m I y, I (25) 

3, 



l 21 



lot, s (25) <Dxmmty&mz&m%mmicm 
ztizmwt? % one**). ***)s*i&aiiEUfcH 

(2 5) *fflV^T»PiB«0**J')S**SiE-r*-'<9 

[0 0 9 2] &E, m&£&ffl2 2 3H7<5;*— *JWU 
g|52 2 2E*&»fflSftfc*:te!)£;M§jE/*7*-** 
BE, 2£i|«&££8££lT£&BjEBB«ffr£-f 

5o a{*«ew\ gnaoMiom (x, y) fc^js-r 30 
saamoiM (x. , y. ) ^(25) es^t 

fWU ftjf^nrcffifil (x. , y, ) ©iff&EJottS 
B£ffi££EBB (X. Y) K43l*SBBB*ftnB* 

E*9ftj&r*. *3s, coBHBBtta-ajfflHHS* 

[0 0 9 3] &.±<D£ 5 E, 4&9§Q%ffi©JgB 1 Eft 

sB&iaggBEj:ft{f» sv>e^*< tfe-gp^a* 

0 £ 5 £ 5 Ktt9ftifitf 2 «W±»«* tifc i: *Ett % 
*©SiB£ftfcffiS£©BBEfctt5&:fet)g#*MiEL 40 

[0 0 9 4] rftt>-6, -hE^BEfc^Ttt^TOBB 
E^TS9tt0^B£B£r«&Bfd:a:< , 
Bl 4E;j?*ft*«k3fc» &¥#©±&£}fl?LfcBfiS! 
fc> ^flE©H*#©**«BLfcB«tf*tttf£»> 

ft¥<*©£ft£jtg LtcmsuDijZgmwmt ltmx 

[009 5] $ fe5*}f iEB«*±E©*SE 50 



B#© A2> t/8*PB#©^;£ ffiB&ffc Jn§/c 
tHfax hfc/Jv£ <-rs2i fctf tss. 

[ftS£©J&»2] B18{±, *f£I*ll©flfi&©J&8g 2 Eft 

5 B#$aa73 & & tf BB&agfifcSi wr s ft &© b t* 

£3„ Bl 8 (a) E^fnSiSE, H-©&¥ftffi 
PL<D»ihBB*\ B^O— SPEfcVT^n^n*ffi-r 
5 J: 3 Effl»«D3ffil d i ~dk frS>»££*xfdl£E:}3 

*HM©^EftsBfiK5aasB6©«imcov> 

[00 9 6] C£T\ Bl 8 (a) E*£ftS«k5E, 
^[rJD„ jb^ffl^-TSil tEcfct)H^i mn (n= 1. 
2--, j, • •, k) imttu MxlfB^i m 1 ZM® 

t)»6nfcBBi mj £H&i m (j + 1) (l<j^ 

k - 1 ) fcowTttt-n^nauBJa^fisfc-rso 

[0 0 9 7] ^LT, *Hfifi0^2Eft5B»«iaS 
■Ciol->T(is Bl 8 (b) E^?n5J:3E, S^B 
^i: LTS^^n/cV^-fti^— ocDB^ i mj EB^-f 
5*aEflfi©B^Wt)^t>?n, ^figBBlMCA^ 

[0 0 9 8] Bl 9tt> *Hfifi©}g«|2Eft5B^S!La 
^«6£0Sfig^-rB^*5o Bl 9E3?Sn«J:3 
E, B^MSSfi6{±, H 3 E^«tife±l^6*OJBB 

i fcftsBisjaasa i aHaft*j«*#r s*^, b^ 
s^MiEgp 2 2 t) EB##im 2 4 mix 



(13) 

23 

T£>o 

[0 0 9 9] 02Oii, 01 9»C^^nfc*HifiO^ 
2 KffiSB&fttiltttt 6 ©ijflFSrjjvT? n-^+- hT* 

r% ±jsim : e-\ : imft<Dzi-'v7wm%m%-rzm io 

K<WBS?tt, a-fOWSt- FWg*- 1 0 4 

**ff-r*ii tic* testis. &&ub€-kb£8 
1 9iu, a^aaa 1 v^mcmmz^- kiss 

*-l 0 4W5n5. flu sa&t-KRsau 9 

[0 10 0] j eLTI3 5(c^$ns<t'5{i:, Xf-y^S 
tt<DX+ <y 7WMiHH£-£ti&. 

[oioi] xx-y^s ncfci^T&is'jfiiE* 

-If tta^gp l l «k 9 &¥&ffi P L < i: t rfe 
»RU B&M3gH6fctt»¥f*»#BD&$tt5o 

IC #4 OBfik tfftg « ft 5 £Ktf $ 5 , 
[0 10 2] Xx-y:/S 3fCfcl>Tft^ft§!<DA 30 

n^o a— «ffc«fc*a»»7©»jSfc«fct)«5# 

iOAA^Tl/fc^Cti, Xr-y^S 4'\jtt?„ * 
fe* ±I2»4Sig ; e- 1 o 4 *t 

»«*7%^"r S fc*6<DX7 >y ^£S"Jj£^ttT t <}; v\ 
[0 10 3] ^Lt> ±IBO«fe 5 K$¥#©JSKtflS7 
Xf'y^S 4WT(cfev>T, ^$ft<DB^£l£ 
0-&*fe^«H«*toS-r*»ffK:X*o tfXfyy' 40 

£SiiR-r£ 0 c©ttrs^5nrcB^±sSip 

BfiT'&'K J-xTtCfeV^Ttt^lt LT0 1 8{il^^n5 

b<s i m j &wm&k Lxmststatrnmco^rK 
oK:fcv^&sfti>*\ a*PB«&£gP2o©«jg&tf 



1#gB2 0 0 1-2 50 1 1 4 
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[0 10 4] Xr-y^S 5K*JV>T, 01 8 

(a) c^«nfciw»r«^iRi3^6»»?nfeBiK»» 

t%t)»ni:Ii (n+1) (l<n^k-l)fc 

mas 2 nctotrfcnSo ciiT% 3*js^aigB2 i©*g 

[o i o 5] ^-lt, m$i&ttti&&<Dmmtj i i&TTz 
3ffiaas»Lfc±T»B«^i!iD^fe«txs. cc-e, $ 

^TBBB«fr5»J*fWf* tmc, X-r-y^S 7Kfi 
v^TJjaiS0^to"e:t«k0**B«« t 4j«*n*. c© 
J; 5 *B»B«fi5>JOBtiJRtf&jaB«04i«tt, B& 
#^B2 4fcj:t>||fT£ft5o WTfcfc^T* c<DB^ 

[0 10 6] 02 Hi, 01 9 fc^*ft;fcB«£j£W 2 
4 0M*Stit*^So 02 1 {C^tl-Si^lCv B 

fl&offi 2 4 u»K9&j?tt$ 2 3 1 tmsmmz 3 
2 1 *^tr. c c r% tivmamum 2 3 1 M&PB& 

2 3 2 ttgigB«tS£g|5 2 0 7 U-A * U 1 5Rtf 

BK£&jnij&2 3 1 icmmztiZo 

[0 10 7] *-LT\ *f^^Wttia52 3 1 «40J^± 

-l) flOB«J*IBIfc*tt5B««SifT5>l*tHIU t 
[0 10 8] .k^mftWtcttx 0 1 8(c^$n-5J:7fc 

iiimn (n=l~k-l) fr^liim (n+1) 
'NOBKatftff M* B . , $ fcB^ imnfr ?>B^ i m 
j ^©BKffiftff 

[0 10 9] 

[»2 5] 

Bn-*j 

<E>o 

[0 110] 
[»2 6] 



(14) 



25 
' j-i 
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Sr-J = 



lV|«j / 



(26) 



(n> j) 



c mmmm2 32 a, ^^gttsgp2 3 1 \c 

J:D»m?nfcS (2 6) KJ^^ftsf^g^j 



*mw (x, y) tc^is-rs, ®mniasvz>&m%&ffi 
<vmm (x, y ) ^SKS^tm-tSo 
[0111] 



X 












1 


i 


.1. 



fCfctt^S ^&f£<7> * 10 CtSt 2 7 ] 



(27) 



fflLsti, 5£ (2 7) KfcttSfejS©?^ WKDSSH 

mm. (x, y ) mmostfzwMmitmic, (x, 
y) <Dmmm*ffimim\z£9&%.?z>o ^lt, 
M^^sti^-^ffipaa^ b - x 7 v -< >mm&% z <d 

CO 1 1 2] VLtOZ 1 )^ #§lffi©^<i£#3B#£a 

#BPL^2ftjj(±ffl^L, wmLrcwmmmzttLT 
mvwmzwi $ tic tmvmvLvfc 

LMffivmm 3 ] ±.nd.nm<DBM 1 ru 2 ic&%mm& 

[0 113] 02 2ti, *^OHSS<DJgffi3fC^5H 

^^hfciasojBji 1 effisnftoasB 1 tmm&m 

C01 14] BflWW!yi«iK8K#snfc'>i'y 

£-1 1 3RZ>V>#7x-X 1 6(i±Wg|5l 4 {Cg 

[0 115] ^?fc, #&!8©llffi©^3C<£5BMi 



[0 116] -73> XT'^S nct3l/>T**3t)ffiiE^: 
-KtfgflSnSfc, Xf77S 2-\iltf„ f LTXr 
•y7°S 2{cfc^Tti\ £*5»3S&©*ifIEttgli:-t5B 

5 0 C c T-S3§B<£K/gg|3 2 0 ti, 02 4fc^ti5J: 
a *— V;l/+- 2 0 4 2 0 3 t*^ts* 

Jl^^tiSo cct*. mx.li=L— !f(i*-v;l/*-2 0 

fcfc J: t>T» j^0&SfflR££*|!Wtt 
[0 117] ^c, Xf>y7S 3fc*J^T:i— tf(i'>% 

< 1 1, 2 tsax±o«?ft«fl!)aue*iisft-r *. ** c © 

Si£JS*^5o fit, B^ffl»-r5fifc->+^^— 
1 1 3fr5£ttffliflSi 4^a^M^*^^n§ci:{i: 
J;oT, ±$i«i 4(crt^^n?c*^>^*w>'^U 

r% mmw®wtfe&2 o«fc o±E»PH»«3£fi*^r 
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ftfc±BB®MB«S5£« t ±EWM»3effll«*W ffl t 

[0 1 18] ±EJt«t*V^TH#Offl*<-a 
«>Cn?&5#&f;:«\ 5^J1#, )IS]gP2 6 

^>?7x-X 1 6^il»Jfi^*^|&«n, 02 5fcjS 
Sft£<k3fc, KfflfcM#fcJSCT-l'>*7x-* 1 6 10 

7 7^2" 2 4 lrt<DK^ft#3 5 BO-OS?* 
-£2 4 2tfj&*T£*iSo ft^>T> fBK^Ka— ifct 

-> y#A>%mz<? 5 c i: #fck o Tff o T 
Co l l 9] ^KXryT's 6^\jtCy*V X-ryT'S 4 

07f7^S6lC^m $¥#©8^ (&¥f*fi{l! 
OA*) W7Lfc*»Sfr#1Wr**U »7L»V^« 20 
W*ftfc»£l«:B:;*Ty7'S3'\B5*o »7"t* 
tWK«nfc»&KttXf- * 7° S 7^31^0 CCT\ a 

»<WR7tta-ifo»^fc:*Si:T ! PJtt*tu K»*a 

xaa^7©fc«>fcs^e.nfcX'i'«y^*^f 

[0 12 0] 7Ttt»PH*Otf'Kfe 

fc^TSBr£B«©*K*v>Tti±i2#Bj&i:ll-@ 
3i^)S«lflSn§. c©*5ftraMifc*fiS 30 

tens. ft*. K»js^wgi52 1 ©«fiRRtf»ff it, ± 

[0 12 1] fit, Xx->7S 8tC*3l/->TS^iB#0 

Xr-yT'S 9fcM">T±R2/^*— ^^SC^feD 
5#*ffiIELfcB«*^LT«ifis*»7 , tS. **, 

jftiBttS&ttiEtt 2 2 fc «fc 0 fTfrn* *\ u OBBS 
^MlEgP 2 2 0«j£&tfKlft (i±taHffl©Jg« 1 40 

SHUT'S 3 » 

[0 12 2] W±©<k5ti:, *£ffiojg&3fi:&3B£ 

«raaai8Kinif, £EBfc£¥Bft££B*B£r 

5 c t (c «}: 0 , a— •ffcfcSBBBKfcfcVT, Steffi 

gP2 6 1 j: oTa— ^(cji^j^n^ rc tb, 3.-mmm 

WHCA*J:3fftc:ait*a3Ci:*«T#*. 

ic, ^BBoBB-SXfcS^S'escfctfT?**. 50 
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[0 12 3] fcfe, a»gP2 6lc«}:D£iMlftffi£ffl0) 
RSlcjS tra-lf '\®PBBO»i&*iI*n-r SSffi 
»i» ±I2HS6O0® 2 tc&3> B£ffl9£B 6 fc t, JSffl Rl 

[HffiOJBfi 4 ] ±£^0Jgttfi:ttSB««i3!gB 

tt, aMM»K:^**ifc#-03te*3R*8«««5<li:R: 

[0 12 4] 02 6H, *%W©HSSO^Il4tt:^5B 
®^ag«4 0O«^5:^-rBT*S5o B2 6tc^n 
#£M<Dl&n 4 fcgj^BB&SgS 4 0 it, 

0 3 t^^n/inssojgji i ic&sB^ggs 1 1 
»»«wi**r*««, bbau natz^<D^mi i 

a. i i B£-&trj&TMggT£t>©T?fc*<> 
[0 12 5] «Tfcfcv*T, *^ffi©^fc^5B#® 

as»©i6f^«rB 2 7 07 d-^ +• - b zpm L-D-Dm 

X^-y^^K^a^-TSa^-Fi:, a^LfcBB© 
^fe t> S^^-ffiE L feBB*4JS"T 5 £*; •? ffiiEt- F 

a^- KRje*-%l*fN"5 C t tc J: <o ^r^n^o ^ 
fcfi^t- F»^» 1 9 fc UT, B^SSB 4 0 <D# 

ftfctta^-Kia^-i 0 4tf»»6h5. fiu 

- F7 x 7X« V 7 h 7 x 7^T*«^t LT «>£</">. 
[0 12 6] ^-LT02 7fC^^tl5j;dtC > Xr>y7 

fetj k> ffli^Er- paws n-f a**- Y«ms& 

[0 12 7] -15, Xf'^S 1 (Cfe^TSfetJffiiE^ 
- F#38?2n* tXf'^S2 'Njt*. S¥B^SS 

b^^k^ s n § „ cc T*a*B#ssap 2 0 ic a , 0 

2 8K^?ns<j;3tCx ±*(R]X^D-;l/+-2 0 1R 
tfT75TlRlX^n-;l/+- 2 0 2 2 0 3 

fnsc a^sui 7 tea ra^H^ssj tv^o 

1 A, l l BTaHBSnfeH«*±KS*PH«kf*ft» 

[0 12 8] -TSi:, 3.— »f{i±7a|^X^D-^-2 

0 iX«T73iRix^a-;^-2 o 2££gftf3;:fcfc: 

-to, 0'JxfiHftjgT^ns^'i'v^^^aji 7± 
T-fijfru )"6^i i A%s»?-r5 r^^^u 
^> *sv«u3te*3gi i B>&ss?-r^ r*p«5 2j t 
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*Vrnfr0ft^£ft£Lfc«i*T-i*£*- 203^ 

2 0 fr^zum® 1 a ^{g^ns, 

#<DM&*'n?o C(DtM&^i 7 tea, ifjiflSian 

[0 13 0] ^LXXy-y-fS 4tl3V^T(±, S3 

»fS^mgP2 i(c«ttiiifT^ni.*v c o^fs^tuap 2 

[0 13 1] 7>tv7S 5iCt3^?M i m®&<D$> 

JJB/<5 ^ - * cDgftt&tffcfc K> ^SrffilE b fcH^© 
3b£ttB^#liIEg|52 2\zHK>m : il£lrL%t>K £<DW 

mm^-ms.^ 2 2 <d& &mjmmt±.m$mv>fcm 1 © 

[0 13 2] &L±<D£5\C. *HSS£D^«l4lC^5iS# 

®as«4 oictntf, mm&4 1 ttft^££'>&< t 

1 2 ofil*. S ft: 46, a— )f tt 1 ®(DM&mmz J: D ±12 

^fSot^T-tSo *fc> mmLrc^micis^rm 30 

•So 

[0 13 3] $fc, 02 6K^2ft;rdf§jg|H 1 KliJfe 

B6£il*e>nT&<U^£ti^93:T?t>&</\, 40 
[0134] ±C±T«MfeW©«l8»c««ii<Wfl 

M-K-r^xi'^©I21S^a j ^CD-ROM#co 

B#^J£$2 0, *fjS&fi}$2 K Btt£3rifiEtf2 2 
ft^££j£V2 4«tfllfikffi 11,41 tttSUeDE 

&mtmmic3s?Lt>tiZo 50 
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[0 13 5] £ fc, fcfJ&&mgP2 1 T*tt\ ffil8&fcj:3 
(2 0) fcjSSnfcflESBHftfclB&ft-f* ffi®>< 

» ufc«»-pjMB«nfcii«iT** c ttfifs u\, je- 
st, fcTRcfc^Ttt, «il&»PB#!£gKiWfi:3iiR 
•TSB^ffla^rii:, gDEr&££frr3H&Ja9£Bfc 

[0 1 3 6] 02 9 a\ *^©HSSo^5{c^«B 

9tc^$n§«fc9^, *^fig©jg®5fc^5B«i!asi^ 
b 5 oti, 03 nfcus&cDjgjg 1 {c^sB^aia 
»Bi 2:pw««ijs«frr«#, SJPB®issaJ2 00 

f-^to tcS¥ft«S«W^gP 2 5 ££9H«gHi»RflS 2 
[0 13 7] CCT, »^ft^WSg|52 5Rt;»JpB 

ftanaft&2 7«, *n^n±»w»i 4tcs^? 
§P2 7tcggi?n5o s?PB#eRia«gP2 7<o 

mmt, BBS9BiEaS2 2ic8gR$nS. 
[0 13 8] *S!SS<Ofl^5{c&2.Bt$!!SgB 
5 0^^:^3^^T^» 04(c^n5J;9tc, ±f2H5g^ 

tt i icfts Bft&e&B i trait^^t-rs cmt 

[0 13 9] JJlTtlSVTs *^BfiO}gfii5{cftSBB 
JaSSB 5 0 <0®ft%. H3 0CSSftft7n-f + - 

sB^^aa^B 5 o a, ±mmm<Dj&m 1 fcft^BBsa 

[0 14 0] S-fXx^^S lfcfcVT, a.— tftci 
[0 14 1] Xr y7°S 1 fcfcv^Ta-lffc.fcO 

'Nitty, ^-lt, xf'^ys 2tcfcv^Ttt, a®gpi 1 

[0 14 2] ^C, XT-yfS 3C*l>Tl» 
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ft*. -73, a— »ffcJ:*llK»70»^KJ:»)«^i* 
*07Jj*»7Lfc«£lCti\ X-r-y:/S 4^jitf 0 
[0 14 3] *LT. Xf7/S 4T*t±, *ff&&fflgP 2 
1 K J: 0«?#«O*fc*V^Tl5ajSifl«ttUl*ft, 

-f, Xr-y/S 5tc43^T, £*BJ»aKl«Rtt2 7fi 

CO 1 4 43 &4s, ±EXttO^ 1 Icfl&mftatt 
BlfcHSfC, X-r-y^S 6K6v>Ttt. Xr-yT'S 5 
JcfcV>TSS??ftfc»Pil«fc^LT*fet)S**?fjE 
"TSfcii©^;*— *£SWU Xf'y7"S 7fc43VT 

m*u ftff*»7r*. 20 

Co l 4 5] ±i5S^iS# 
fcLTtt, J§&Sftfc£S©B&©*-C«¥#ffitfgfe 

T, 03 1 *#B8t"30SMM-«. 
CO 1 4 6] H 2 9 fcjasSftfeH*£**f iE* 2 2 tt» 

±E0«k3lcS (2 5) fc«kS*«a»*SWf"r*£:i: 30 
■T5ftii>**0ft*^y|IWD»cl86ft» froS^ftffi 

CO 1 4 7] 03 1 (a) (u*;£ft«J:3lc % £4398 
■P^ft*«»ej:0«fi»ffi3 l'sSKSftS. 4 

Co 1 4 8] -73, 03 1 (b) fc^£ ft £><$:?{;:, m 



3 2 ©£Jffi£G§<Dji«±, 03 1 (a) fcaUrft* 

a x + by + c = 0 (a 2 + b* = 1) 

cct*, ±es (2 8) <\<D%Titmt. matt-fe^ 

H*OSTti««WR7 , r*4:, KftQA£*8f<?;( 50 
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fc&^SftSo ^LT, &£M&K,J; s&mbmom 

mmff&TTZffimt, 03 1 (a) ©if^KJtLT* 

CO 1 4 9] ftot, W±<fc!?, ##/|v£v*K 
if B«^#*fIEgp 2 2t«t5ffi8^g|tcSHUfcfi?#a 

co i so] sfc, 0 2 9 it ^ft^tr^WRMflsv 

( a ) mSfi&J^Bttfflj&a <k a KBfiLhT?* ?x 
^U>y%t?-573a, (b) t^h^7AKJ:*««» 

(d) f-*x*-*#&K %£fl!)««»iKia*<affl«ft 

CO 1 5 1] CCT\ *%WOllSfiC!5)B^5C 
&SSB«, H3 2fc35Sti*J:3a«lJ«t"r*ci:t) 
T*#So T&fc>%, 03 2{c^£ftftB#$0 ! SIgB5 1 

t±, 029 K^^nftBfSiaagg sot mwteffii$.z 
co 1 5 2] -use^c, jzmzttsbttzm^faffiicts 

/^-ytf, g&Sftt£8ofiilitt/S£-;y£LTB 

it^?n%o ^ot, n»csa&*ftfcjiistt/<* 

- > © IS) t © {f 6 O # ^ * c 1 1 J: !7 ^fe 43 1> ^ <D 

^ 1 1 TS^-T * C t So 
CO 1 5 3] CCT', 03 2K.7TiZtircWMWW—> 
^mSI3 2 8(i, ffl^^ftfcMS[(DB#(C4oV>Ta^tt^ 

co 1 5 4] t-f> KaftoaftK^vTH^Ksc 

h fc^-f J L, ^ ft^PftcDB^-fe 9* y h ^J-XT©S 

(28) ic7*2ti?>mi@.ijmmc%Tii.tbz> a 

(2 8) 

-^T'SS (a, b) <Dffe>O#^J6§ 0 COidtC 
LT, ±S^a©B«^TK:WLT (a, b) <D/^* 



(18) 



33 



A. 



Sit LT3IS?-f fctfr-ts, 

[0 1 5 5] X¥HftO@nSjRlc43^TH:, U 
-fSo Ex>y^B&rt©£T©j5fcftLTHough 

[0 1 5 6] 03 3t±> 03 3 (a) fc^£ftfc:B<iS 

mfr%m3 3 (b) (c^nse-p^K'NOHough^ 

&£SSHI§-f 30T*&3 O 03 St^^n^J;^^ Houg 
hffiftt * 0 BflLhO** P 1 ~ P 3 H\ ^n^f 4a*f JS 
Ll~L3C^n5, f LT, <1©,£5& 
lLk©£j&fco^Tj|fT-f3i:, 0-p£IBK 

cl©j&C P{iX«yi^M^C^V>T0< ©X'^*MI5 20 

p) t-rntf, ^<ds (29) t»K-r*ii»««ttffi* 

p=xcos0+ysin0 (2 9) 

fix, ±iat»©H«^TK»L, KfirtStfiwr 

C P ££&JfiW U cne>©£tci>tt3 0©lfS>o 

6o**»i*ci:Ai«-e*5o ;:©J:?{;:LT, fifeffig! 
©lRjt©fcre>-o*jbVjM£fcB&£> gKuftfcX^Bftt 
LTjWWfttffiVo 30 

[0 15 7] 03 4(4. #5glB©IIS6©J$fi5fC&5B 
®ffl3gg£fcttS£=©1i$(W£^T0T*&5o 0 3 

BW\ 02 9fcwSftifc»?fl*l«flJJtS52 5^, 03 

[0 15 8] ±8B»£«ttB»2 1 fc«fcD8tHSnfcftai 
jSi:»J6jSiOfiH:»|g^WfWB, -T^^^SfcDM^ 

aX+bY + cZ + d = 0 (a 2 + b* 
fcfcS, 3£W±©3#7E^l£:ffl^T, ±E S FB7J8 
SfcfcttS 4O0/<7^- £ (a, b, c, d) fc||/J\ 
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[0 15 9] *fc, ±e©ffiHffifcS-3<#JS«Hifcfc 
•Bit WfMWK& < ^IctxT^S Rfliltttf »i^© 
[0 16 0] 03 5tt #fgti©HSg©^5K:&3B 

iiwasrawK43it5SEaoflija0!*^rHT**. 03 
5 testis <fc 5 *nss©mi 5 icfl&piftfias 

Btt, H2 9tiS«nfc*^#fli«¥!liiff2 5^ 03 
2fc^SftfcB«K^*-}'tJttB»2 8©ftfc>t>£, ¥ 
ffitr$!lg&2 9 C 1 1 LTfcfiV. 

[0 16 1] CUT, ¥SH"«2 9 &B$©Ji» 
RffcfctfSSHMPl He ft? s K?#4>Ae*lffir 

So 03 6 W\ 03 5{C^£*lfc¥ffifrffi'Jgl52 9©ft$ 

wt^-rHT**. 03 GKTj&nz&oK, w-mam 

g|52 9«X#<y hftig2 7 1 §ft*?2 7 2 i:, 3 

&75*aastBa&2 7 3 ¥x*ibtf2 7 4 *v 

fr5>&37frg2 7 1 at, ^Un>=^-^(D^1E55 
-2 7 1 b t, j£S55-2 7 1 b<DWl%*toffltZm 
ftgP2 7 1 c ttttS. 
[0 16 2] £ZLT\ ±I3X/Hy bytffi.2 7 1 fc:|3V>T 
fi> %M2 7 1 afC<£?)f§£;*nfcX#>y 
ffiP LtSS.fc'plC, ^4=7-2 7 1 b^Btttf 2 7 

1 ct«t»)l6!W*n*o Sfe, §7fe^?2 7 2« > 

>y h^2 7 1 lcttt5&S&¥ibmfe-£tlZ\,>zmm 
tSBStlfcPSD (Position sensitive detector) 
■f C C D^OJWt^HK? K J: 0*J««tU «¥ftffiP 

L*»6©s«3e©isj**ttar*. ^ a^gpi nc 
£s*ifc3Wtasyii? 1 1 4 *, ±aaK*»t*swie*f 

2 7 2 ^LTfilfflLTIxtV^ 

[0 16 3] 3 ^7C^S»mg|5 2 7 3(i, X^K-y 

3t»2 7 1 t%ftm?2 7 2 Rtf^ftig 
?2 7 2tftjWJLfeS«JtO|ftlflC«:UT» H^SIfiO 

Jia*fflvSc:i:fc«J:»5B»iDaS»5 3*Si(if:Lft 

S§ftffiPL0 335?7ca« (X, Y, Z) ^»WT5o 
*LT, ¥mW&$t 2 7 4 tt, 3 *7CitmjWl» 2 7 3 
fc J: l))|Cffl«nfera-ia8±K*V^ 3 jSftUO 3*7ME 

+ c 2 =1, c>0) (3 0) 

S (3 l) (cfcOtrS^tiSo 

$ = cos c (3 1) 

lot, WBSftfc«»©H«£TK»LT»5#E<0 
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[0 16 4] tom, B^fl*ffiiEg|5 2 2<Dmmi03^ 

I^TiS (14) &^LiS (18) £*fTLTJ6¥#<D|n] 

■5tc, ID— co^^^ffiP Lfcfc^-i.iiiJ:H#*\ B§s© 

g-rSCfcfCkDHf&i m j (1 ^ j SK) tfff&tu 
mMim 1 i:H® i m 2 ©<£ o KRO^^IrKC 
fcVTa^-fSeifcJC.fcOfi ZtiKW&i m j fcBIti 

m (j + i) (i^j^K-i) tomTHzztiznw. 
£-t> n^«w ^ n 5 o 

[0 16 5] 03 7W\ *fm<0$3&<DBM6\Z%5M 

msM^m6 0<om^mrmT'$>^ o 03 7tc*?n 

-5 J; 5 fc, *HfigO^ 6 c^^B^saasa 6 0 fcfc, 

m®?mi§&m-t%i)\ mmmmm.^2o<DHt>oiz 

•So 30 

[0 16 6] JJ20<fc3*«te*#r***tt0}B8&6 

fcmzmmwmmw 6 o a, mm<omm 2 efl&MMB 

SSB6t^ci)ff-r5^ JetTKJs^T, #HBS© 
6 C^SB^jiaaSB 6 0 ©ftlff 03 8 {C,72 

[0 16 7] S-fX-r^ys 1 fcfeVT, a-+ffc«fc 

Xf'^S 1 C\it#, K^.— f fc <fc 0 WlWbTst V •/ 40 
[0 16 8] — 77> HCj3V>Ta— iffCfct) 
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